發(fā)布時間：2018-06-13 13:20

  本文選題：HDFS + BT�。� 參考：《吉林大學》2013年碩士論文

【摘要】：隨著計算機與網(wǎng)絡的迅猛發(fā)展，數(shù)據(jù)量也與日激增，據(jù)思科在其《全球云數(shù)據(jù)報告中》指出，由于用戶和企業(yè)的不受限制地訪問及應用數(shù)據(jù)的需求，在2010至2015年期間，全球云數(shù)據(jù)流量將以每年66%的速度增長。傳統(tǒng)的分布式高性能處理平臺處理數(shù)據(jù)的能力已經(jīng)滿足不了井噴式增長的數(shù)據(jù)處理請求。應運而生的云計算、云存儲則滿足此類數(shù)據(jù)密集性的服務請求。 本文研究內(nèi)容是基于Hadoop的云系列服務系統(tǒng)的文件系統(tǒng)HDFS在其讀寫過程中傳輸策略的研究。旨在通過對HDFS讀寫等基本操作的效率提高及容錯來實現(xiàn)更高效率及高度容錯的文件系統(tǒng)。云存儲中的每個操作都離不開對文件系統(tǒng)的調用，對文件系統(tǒng)HDFS中最基本、最常用的讀寫操作的傳輸過程進行研究并改進，實現(xiàn)并行讀寫及高度容錯，這會極大程度提高云存儲服務中數(shù)據(jù)的訪問速度及可用性問題。 本文首先介紹了云存儲的相關理論和技術，闡釋了云存儲的定義，并對其應用場合加以描述。緊跟著又對HDFS進行了透徹的剖析，并對其相關技術進行了解析和比較。隨后對HDFS讀寫容錯需要的技術進行詳盡的描述，為接下來的研究提供的良好的技術保障。最后在通過研究HDFS讀寫機制進行分析之后，對HDFS的讀寫機制中文件傳輸進行改進，實現(xiàn)文件數(shù)據(jù)塊級的并行傳輸功能。從而為云存儲的高延遲及副本安全性問題提供一個可靠的解決方案。 本文主要實現(xiàn)了云存儲文件系統(tǒng)的文件系統(tǒng)HDFS中文件讀寫過程中的并行傳輸策略，及改進的副本自動復制策略，提高了讀寫效率，降低延遲時間，為云存儲用戶提供高效并穩(wěn)定的服務。在本文的改進策略下，充分利用了副本的存在，分散了網(wǎng)絡的負載，，數(shù)據(jù)讀的效率可提高160%，數(shù)據(jù)副本的復制效率也大大提高。
[Abstract]:With the rapid development of computers and networks, the amount of data is also increasing, according to Cisco in its Global Cloud data report, due to the unrestricted access and application data needs of users and enterprises, between 2010 and 2015, Global cloud data traffic will grow at an annual rate of 66%. The traditional distributed high performance processing platform can not meet the data processing requirements of blowout growth. Cloud computing comes into being and cloud storage meets such data-intensive service requests. This paper focuses on the research of HDFS file system based on Hadoop in the process of reading and writing. The purpose of this paper is to achieve a more efficient and highly fault-tolerant file system by improving the efficiency and fault tolerance of basic operations such as reading and writing HDFS. Every operation in cloud storage can not be separated from the call of file system. The transmission process of the most basic and commonly used read and write operations in file system HDFS is studied and improved to realize parallel reading and writing and highly fault-tolerant. This greatly improves the speed and availability of data access in cloud storage services. This paper first introduces the theory and technology of cloud storage, explains the definition of cloud storage, and describes its application. Then the HDFS is analyzed thoroughly, and the related technology is analyzed and compared. Then the technology needed for reading and writing fault tolerance in HDFS is described in detail, which provides a good technical guarantee for the following research. Finally, after analyzing the reading and writing mechanism of HDFS, the paper improves the file transmission in HDFS reading and writing mechanism, and realizes the parallel transfer function of file data block level. This provides a reliable solution for high latency and replica security problems in cloud storage. In this paper, the parallel transmission strategy in HDFS file system and the improved replica automatic replication strategy are implemented, which can improve the efficiency of reading and writing and reduce the delay time. Provide efficient and stable services for cloud storage users. Under the improved strategy of this paper, we make full use of the existence of replicas and spread the load of the network. The efficiency of data reading can be improved by 160%, and the efficiency of replication of data replicas can also be greatly improved.
【學位授予單位】：吉林大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333;TP309

【參考文獻】

相關期刊論文 前3條

1 樊景超;周國民;;基于Lucene的“農(nóng)搜”并行索引技術研究[J];農(nóng)業(yè)網(wǎng)絡信息;2009年08期

2 吳晟;蘇慶堂;羅斌;趙莉楠;蔡燦民;;基于Socket和多線程技術的并發(fā)服務器的研究[J];昆明理工大學學報(理工版);2006年04期

3 張樹剛;張遂南;黃士坦;;CRC校驗碼并行計算的FPGA實現(xiàn)[J];計算機技術與發(fā)展;2007年02期

相關博士學位論文 前1條

1 宋瑋;分布式存儲系統(tǒng)中的節(jié)點自主性問題研究[D];華南理工大學;2010年

相關碩士學位論文 前2條

1 張倩;基于Linux操作系統(tǒng)的嵌入式視頻監(jiān)控系統(tǒng)的研究與開發(fā)[D];天津工業(yè)大學;2007年

2 陳華;Linux下BT客戶端的設計與實現(xiàn)[D];蘇州大學;2009年

本文編號：2014151