元數(shù)據(jù)存儲單元設計與實現(xiàn)
發(fā)布時間：2018-07-18 16:27
【摘要】：隨著信息的大發(fā)展數(shù)據(jù)呈現(xiàn)爆炸式增長，用于存儲飛速增長的數(shù)據(jù)的分布式文件系統(tǒng)應運而生。目前主流的分布式文件系統(tǒng)多采用元數(shù)據(jù)與數(shù)據(jù)分離的管理機制，即將文件元數(shù)據(jù)和命名空間保存到元數(shù)據(jù)存儲節(jié)點上，而向系統(tǒng)上傳的數(shù)據(jù)則保存到數(shù)據(jù)節(jié)點上，以便高效、高擴展、安全、快速地存取數(shù)據(jù)。因此，，在整個分布式文件系統(tǒng)中，元數(shù)據(jù)的存儲占據(jù)舉足輕重的地位。 本文主要研究分布式文件系統(tǒng)中的元數(shù)據(jù)存儲單元。通過對元數(shù)據(jù)高可用性技術的深入研究，在前人研究成果基礎上，設計實現(xiàn)了Cstore分布式文件系統(tǒng)元數(shù)據(jù)存儲單元，并對實現(xiàn)的存儲單元進行了部分性能測試。主要工作如下： 研究了分布式文件系統(tǒng)中目前主流的文件系統(tǒng)以及這些系統(tǒng)的設計理念與設計原則，并對上述主流的文件系統(tǒng)的元數(shù)據(jù)管理的方式方法進行了重點研究，同時對元數(shù)據(jù)職責和元數(shù)據(jù)管理的方式進行了詳細的闡述。 研究了分布式文件系統(tǒng)常用的系統(tǒng)組織模式，并詳細的介紹了兩大類設計模式：active/active模式和active/standby模式。對這兩大類設計模式介紹的同時分析了這兩大模式下的不同子模式的優(yōu)點與缺點。同時介紹了常用的服務器編程I/O模型。 對Cstore文件系統(tǒng)進行了概括性的介紹，并對整個系統(tǒng)中所涉及到的關鍵技術進行了介紹與闡述，其中對元數(shù)據(jù)存儲單元提供服務所面臨到的問題提出了解決方案。如對數(shù)據(jù)安全和穩(wěn)定服務方面提出的主從互備機制，面對如何確保用戶上傳的元數(shù)據(jù)快速定位到用戶上傳元數(shù)據(jù)定位元數(shù)據(jù)服務器，上傳的文件如何快速定位到需要上傳數(shù)據(jù)的數(shù)據(jù)服務器等技術。同時介紹了元數(shù)據(jù)規(guī)則服務器用到的一些關鍵技術。 總體介紹上述元數(shù)據(jù)存儲單元的模塊劃分，然后詳細給出各子模塊的設計類圖和數(shù)據(jù)流程圖，并對圖中所涉及到的類的作用加以詳細說明和介紹。 對實現(xiàn)后的元數(shù)據(jù)存儲單元集群進行了測試。結果表明該元數(shù)據(jù)存儲單元集群能夠高效穩(wěn)定地對外提供元數(shù)據(jù)訪問，性能達到了設計要求。
[Abstract]:With the rapid development of information, the distributed file system which is used to store the rapidly increasing data comes into being. At present, the mainstream distributed file system uses the management mechanism of separating metadata from data, that is, file metadata and namespace are saved to metadata storage node, and data uploaded to the system is saved to data node, so as to be efficient. High extension, secure, fast access to data. Therefore, metadata storage plays an important role in the whole distributed file system. This paper focuses on metadata storage units in distributed file systems. Based on the previous research results, the Cstore distributed file system metadata storage unit is designed and implemented, and the performance of the implemented memory unit is tested. The main work is as follows: the main file systems in the distributed file systems are studied, and the design concepts and principles of these systems are also discussed. The methods and methods of metadata management of the main file system mentioned above are studied in detail. At the same time, the responsibility of metadata and the way of metadata management are expounded in detail. This paper studies the common system organization patterns of distributed file systems, and introduces in detail two kinds of design patterns: active / active mode and active/standby mode. In this paper, the advantages and disadvantages of different sub-patterns in these two kinds of design patterns are analyzed. At the same time, I / O model of server programming is introduced. This paper gives a general introduction to Cstore file system, introduces and expounds the key technologies involved in the whole system, and puts forward solutions to the problems faced by metadata storage units in providing services. For example, the master-slave mutual backup mechanism for data security and stability services, how to ensure that the metadata uploaded by the user can be quickly located to the metadata server, Upload files how to quickly locate the need to upload data server and other technologies. At the same time, some key technologies used in metadata rule server are introduced. The module partition of the above metadata storage unit is introduced in general, and then the design class diagram and data flow chart of each sub-module are given in detail, and the functions of the classes involved in the diagram are explained and introduced in detail. The implementation of the metadata storage unit cluster was tested. The results show that the cluster of metadata storage units can provide metadata access efficiently and stably, and the performance meets the design requirements.
【學位授予單位】：電子科技大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333
本文編號：2132486
[Abstract]:With the rapid development of information, the distributed file system which is used to store the rapidly increasing data comes into being. At present, the mainstream distributed file system uses the management mechanism of separating metadata from data, that is, file metadata and namespace are saved to metadata storage node, and data uploaded to the system is saved to data node, so as to be efficient. High extension, secure, fast access to data. Therefore, metadata storage plays an important role in the whole distributed file system. This paper focuses on metadata storage units in distributed file systems. Based on the previous research results, the Cstore distributed file system metadata storage unit is designed and implemented, and the performance of the implemented memory unit is tested. The main work is as follows: the main file systems in the distributed file systems are studied, and the design concepts and principles of these systems are also discussed. The methods and methods of metadata management of the main file system mentioned above are studied in detail. At the same time, the responsibility of metadata and the way of metadata management are expounded in detail. This paper studies the common system organization patterns of distributed file systems, and introduces in detail two kinds of design patterns: active / active mode and active/standby mode. In this paper, the advantages and disadvantages of different sub-patterns in these two kinds of design patterns are analyzed. At the same time, I / O model of server programming is introduced. This paper gives a general introduction to Cstore file system, introduces and expounds the key technologies involved in the whole system, and puts forward solutions to the problems faced by metadata storage units in providing services. For example, the master-slave mutual backup mechanism for data security and stability services, how to ensure that the metadata uploaded by the user can be quickly located to the metadata server, Upload files how to quickly locate the need to upload data server and other technologies. At the same time, some key technologies used in metadata rule server are introduced. The module partition of the above metadata storage unit is introduced in general, and then the design class diagram and data flow chart of each sub-module are given in detail, and the functions of the classes involved in the diagram are explained and introduced in detail. The implementation of the metadata storage unit cluster was tested. The results show that the cluster of metadata storage units can provide metadata access efficiently and stably, and the performance meets the design requirements.
【學位授予單位】：電子科技大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333
【參考文獻】
相關期刊論文 前3條
1 雷江濤;白照高;;網(wǎng)絡存儲技術的發(fā)展、現(xiàn)狀及應用[J];工業(yè)儀表與自動化裝置;2007年01期
2 段翰聰,盧顯良,宋杰;基于EPOLL的單進程事件驅動通信服務器設計與分析[J];計算機應用;2004年10期
3 段翰聰;楊帆;李林;;基于二維表的對象存儲元數(shù)據(jù)管理[J];實驗科學與技術;2012年06期
本文編號：2132486
本文鏈接：http://www.wukwdryxk.cn/kejilunwen/jisuanjikexuelunwen/2132486.html
最近更新
教材專著
