發(fā)布時間：2018-06-10 19:00

  本文選題：分布式存儲系統(tǒng) + 數據一致性�。� 參考：《電子科技大學》2013年碩士論文

【摘要】：隨著國家對下一代寬帶無線移動網絡建設的推進，能夠滿足各種電信業(yè)務數據要求的移動控制網絡對后臺的存儲支持提出了越來越高的要求。傳統(tǒng)的存儲解決方案往往不能滿足系統(tǒng)和用戶對可擴展性和可用性的需求，在面對海量用戶和海量文件數據的情況下更是捉襟見肘：系統(tǒng)規(guī)模無法適應用戶與應用規(guī)模的增長，應對海量數據的處理與訪問也顯得力不從心。 本文首先對國內外幾種主流的分布式文件系統(tǒng)進行了調研與介紹，面對海量用戶與海量文件的應用場景，設計并實現(xiàn)了分布式存儲系統(tǒng)CSTORE中的資源定位規(guī)則和數據存儲模塊。 CSTORE采用純分布式的架構，將元數據與數據的分別存儲在不同的集群中，并實現(xiàn)了一種基于兩級映射的資源定位規(guī)則，結合負載均衡與數據恢復策略，能夠提供較好的可擴展能力。元數據和數據的分離使得客戶端對兩種數據的訪問分開，便于它們各自的管理與性能的優(yōu)化；而集群的利用進一步促進了數據的分布式管理，避免了單點失效帶來的危險。 資源定位規(guī)則建立了資源與存儲數據的服務器之間的映射關系，它為客戶端提供了一種可靠的尋址方法，系統(tǒng)中任何節(jié)點都可以使用它獨立計算出資源的位置，既充分利用了客戶端的資源，又減輕了服務器端的壓力；它的偽隨機特性和映射變換同時也為系統(tǒng)提供了較好的負載均衡和可擴展性。 數據存儲模塊為用戶提供數據存儲與訪問服務，存儲到系統(tǒng)中的數據按資源定位規(guī)則被分為了多個不同的資源管理單元，資源管理單元的多個副本之間兩兩建立心跳連接以推送數據和交換時序，在一定程度上保證了數據的可用性的同時也保證了一致性；容錯方面，冗余機制保證了系統(tǒng)可以容忍在一定范圍內的失效，數據恢復機制可以使得失效的模塊能夠盡快的對客戶端服務。在數據存儲模塊中所有文件片都使用全局唯一ID對所存儲的數據進行了去重處理。數據去重可以節(jié)省存儲和管理成本，提高物理存儲空間利用率。 本文最后對存儲系統(tǒng)的功能和性能進行了測試，證明了它能正確執(zhí)行所需功能并保證了其高效性。
[Abstract]:With the development of the next generation broadband wireless mobile network, the mobile control network, which can meet the data requirements of various telecommunication services, has put forward higher and higher requirements for the storage and support of the background. The traditional storage solutions often can not meet the scalability and usability requirements of the system and users, especially in the face of massive users and massive file data: the scale of the system can not adapt to the growth of users and applications. It is not enough to deal with the massive data processing and accessing. Firstly, this paper investigates and introduces several mainstream distributed file systems at home and abroad, facing the application scenarios of massive users and massive files. The resource location rules and data storage modules in CSTORE are designed and implemented. CSTORE uses a pure distributed architecture to store metadata and data in different clusters. A resource location rule based on two-level mapping is implemented, which combines load balancing and data recovery strategy to provide better scalability. The separation of metadata and data enables clients to access the two kinds of data separately to facilitate their respective management and performance optimization, while the utilization of clusters further promotes the distributed management of data. The resource location rules establish a mapping relationship between the resource and the server that stores data, which provides a reliable addressing method for the client. Any node in the system can use it to calculate the location of the resource independently, which not only makes full use of the resources of the client, but also reduces the pressure of the server. Its pseudorandom characteristics and mapping transformation also provide better load balancing and extensibility for the system. Data storage module provides users with data storage and access services. The data stored in the system is divided into a plurality of different resource management units according to the resource location rules. A heartbeat connection is established between a plurality of copies of the resource management unit to push the data and exchange time series. To a certain extent, the availability of the data is guaranteed, and the consistency is ensured. In the fault-tolerant aspect, the redundancy mechanism ensures that the system can tolerate the failure in a certain range. The data recovery mechanism can enable the invalid module to serve the client as soon as possible. In the data storage module, all the files are de-reprocessed using the global unique ID. Data removal can save storage and management costs and improve the utilization of physical storage space. Finally, the function and performance of the storage system are tested, which proves that it can correctly perform the required functions and ensure its high efficiency.
【學位授予單位】：電子科技大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333

【參考文獻】

相關期刊論文 前5條

1 袁培森;皮德常;;用于內存數據庫的Hash索引的設計與實現(xiàn)[J];計算機工程;2007年18期

2 吳蔚;網絡站點的負載均衡技術探索[J];計算機時代;2004年07期

3 韓德志;陳磊;;存儲網絡高可用系統(tǒng)設計關鍵技術研究[J];計算機應用研究;2007年08期

4 敖莉;舒繼武;李明強;;重復數據刪除技術[J];軟件學報;2010年05期

5 薛軍,李增智,王云嵐;負載均衡技術的發(fā)展[J];小型微型計算機系統(tǒng);2003年12期

，

本文編號：2004264