發(fā)布時(shí)間：2019-01-28 20:22

【摘要】：并行編程是有效利用多核資源的重要方式。然而,傳統(tǒng)的并行編程模型要求程序員手動(dòng)管理諸多的并發(fā)細(xì)節(jié),使得并行編程變得困難且易錯(cuò)。MapReduce編程模型通過(guò)自動(dòng)管理并發(fā)任務(wù),簡(jiǎn)化并行編程。Phoenix是MapReduce模型在多核系統(tǒng)上的一種實(shí)現(xiàn),它證明了應(yīng)用程序通過(guò)使用MapReduce能夠獲得與Pthreads相媲美的性能。但是,Phoenix的可伸縮性較差。本文旨在分析影響Phoenix可伸縮性的關(guān)鍵因素,然后設(shè)計(jì)并實(shí)現(xiàn)一個(gè)具有較好可伸縮性的MapReduce庫(kù)SMR。本文的主要貢獻(xiàn)為:(1)通過(guò)實(shí)驗(yàn)測(cè)試以及使用Linux Perf等性能分析工具,深入分析影響Phoenix可伸縮性與性能的關(guān)鍵因素。Phoenix因Pthreads線程對(duì)共享地址空間的競(jìng)爭(zhēng)導(dǎo)致其較差的可伸縮性,因map和reduce階段間存在同步屏障影響性能。(2)利用課題組已有的新型線程模型Sthread,設(shè)計(jì)并實(shí)現(xiàn)一個(gè)Phoenix接口兼容且具有較好可伸縮性的MapReduce庫(kù)SMR。SMR通過(guò)避免多個(gè)線程對(duì)共享地址空間的競(jìng)爭(zhēng),提升系統(tǒng)的可伸縮性;通過(guò)讓map和reduce階段間的無(wú)阻塞流水線并行,提升系統(tǒng)的性能。(3)在一臺(tái)32核機(jī)器上使用Phoenix測(cè)試程序?qū)�SMR和Phoenix進(jìn)行性能評(píng)估,還分析了近年來(lái)Linux內(nèi)核的優(yōu)化對(duì)多核MapReduce庫(kù)可伸縮性與性能的影響。實(shí)驗(yàn)結(jié)果表明,基于SMR運(yùn)行的五個(gè)測(cè)試用例都具有較好的可伸縮性和性能。32核CPU下,SMR相比Phoenix的性能提升為9.0倍～26.7倍。Linux內(nèi)核的優(yōu)化能夠有效提升Phoenix的執(zhí)行效率,而對(duì)于SMR的影響較小。(4)為了高效支持迭代型MapReduce應(yīng)用,本文初步設(shè)計(jì)并實(shí)現(xiàn)了面向多核可伸縮的迭代式MapReduce庫(kù)iSMR。SMR因每次迭代計(jì)算都需要重復(fù)地創(chuàng)建和銷毀環(huán)境,浪費(fèi)大量的時(shí)間,致使其無(wú)法高效地支持迭代型應(yīng)用。iSMR只在處理流程的開(kāi)始和結(jié)束時(shí)創(chuàng)建和銷毀環(huán)境,并且它采用線程池復(fù)用線程,以提升迭代型應(yīng)用的處理效率。本文提出的SMR為程序員提供了一種簡(jiǎn)單高效的方式使用多核資源。此外,本文關(guān)于多核MapReduce庫(kù)可伸縮性的分析,對(duì)其他Pthreads程序的分析具有指導(dǎo)意義;關(guān)于近年來(lái)Linux內(nèi)核優(yōu)化的分析,對(duì)Linux的相關(guān)研究工作具有參考價(jià)值。
[Abstract]:Parallel programming is an important way to utilize multi-core resources effectively. However, the traditional parallel programming model requires programmers to manage many concurrency details manually, which makes parallel programming difficult and error-prone. The MapReduce programming model manages concurrent tasks automatically. Simplified parallel programming. Phoenix is an implementation of the MapReduce model on multi-core systems, which proves that applications can achieve performance comparable to Pthreads by using MapReduce. However, Phoenix has poor scalability. The purpose of this article is to analyze the key factors that affect the scalability of Phoenix, and then design and implement a MapReduce library SMR. with good scalability. The main contributions of this paper are as follows: (1) through experimental testing and using Linux Perf and other performance analysis tools, Deeply analyze the key factors that affect the scalability and performance of Phoenix. Phoenix has poor scalability due to the competition between Pthreads threads and shared address space. Due to the existence of synchronization barrier between map and reduce phase, the performance is affected. (2) using Sthread, a new thread model, which has been developed by our research group. A Phoenix interface compatible and scalable MapReduce library SMR.SMR is designed and implemented to enhance the scalability of the system by avoiding the competition of multiple threads on the shared address space. The performance of the system is improved by parallel non-blocking pipeline between map and reduce. (3) using Phoenix test program to evaluate the performance of SMR and Phoenix on a 32-core machine. The effects of Linux kernel optimization on the scalability and performance of multi-core MapReduce libraries are also analyzed. The experimental results show that the five test cases based on SMR have good scalability and performance. Under 32 core CPU, the performance of SMR is 9.0 ~ 26.7 times higher than that of Phoenix. The optimization of Linux kernel can effectively improve the execution efficiency of Phoenix. In order to support the iterative MapReduce application efficiently, this paper designs and implements a multi-core scalable iterative MapReduce library iSMR.SMR, which needs to create and destroy the environment repeatedly for each iteration calculation. ISMR only creates and destroys the environment at the beginning and end of the processing process, and it uses thread pool to reuse threads to improve the processing efficiency of iterative applications. The proposed SMR provides programmers with a simple and efficient way to use multicore resources. In addition, the analysis of the scalability of multi-core MapReduce libraries is of guiding significance to the analysis of other Pthreads programs, and the analysis of Linux kernel optimization in recent years has a reference value for the research work of Linux.
【學(xué)位授予單位】：中國(guó)科學(xué)技術(shù)大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP311.1

【相似文獻(xiàn)】

相關(guān)期刊論文 前10條

1 ;A Difference Method of the Gravity Center with Double Pilots for the MC CDMA System[J];The Journal of China Universities of Posts and Telecommunications;2000年04期

2 劉巍;吉祥鳳凰與亮麗彩虹——Phoenix與同濟(jì)大學(xué)共建自動(dòng)化實(shí)驗(yàn)室[J];自動(dòng)化博覽;2002年01期

3 ;刺激1998[J];軟件世界;1998年12期

4 ;搖擺爵士——Phoenix Gold全車音響解決方案[J];音響改裝技術(shù);2009年01期

5 陳子超;;Phoenix瀏覽器[J];軟件導(dǎo)刊;2003年04期

6 ;Phoenix BIOS的ID識(shí)別技術(shù)[J];個(gè)人電腦;2004年08期

7 郝顏;涂碧波;詹劍鋒;孟丹;;Phoenix集群操作系統(tǒng)Web Portal的設(shè)計(jì)與實(shí)現(xiàn)[J];計(jì)算機(jī)工程;2006年15期

8 ;Methods for Industrial Measurement and Basic Principles of Their Choices[J];Geo-Spatial Information Science;2002年02期

9 王欣;;安心使用 Phoenix Recover Pro 6[J];個(gè)人電腦;2006年07期

10 馬天蔚;劉燕之;;終端安全招來(lái)鳳凰——訪Phoenix全球CEO兼總裁Albert E.Sisto[J];每周電腦報(bào);2006年18期

相關(guān)會(huì)議論文 前1條

1 趙忠;楊秀蓮;;Phoenix-100全自動(dòng)微生物鑒定／藥敏系統(tǒng)的臨床應(yīng)用及評(píng)價(jià)[A];第九屆西北五�。▍^(qū)）檢驗(yàn)醫(yī)學(xué)學(xué)術(shù)會(huì)議論文匯編[C];2005年

相關(guān)重要報(bào)紙文章 前8條

1 ;一個(gè)時(shí)代的終結(jié)[N];中國(guó)電腦教育報(bào);2009年

2 金文;力挺3G自有標(biāo)準(zhǔn) 聯(lián)想手機(jī)捷報(bào)頻傳[N];中國(guó)質(zhì)量報(bào);2007年

3 洪培能;采用Phoenix BIOS主板的一個(gè)隱藏功能[N];中國(guó)電腦教育報(bào);2003年

4 Rebios.net 王慶東;Phoenix的FirstBIOS技術(shù)簡(jiǎn)析[N];電腦報(bào);2003年

5 福建 林峰杰;Modem損壞引起黑屏[N];電腦報(bào);2005年

6 飛翔的戰(zhàn)羽;下一代你了解多少[N];中國(guó)電腦教育報(bào);2004年

7 上海 大海馬、小Paul;BIOS控制著什么[N];電腦報(bào);2004年

8 陳國(guó)雄　通訊員　 李品 馬楚煒;地中海PHOENIX首航南沙[N];中國(guó)水運(yùn)報(bào);2007年

相關(guān)碩士學(xué)位論文 前4條

1 向晟;乍得Phoenix工區(qū)潛山優(yōu)質(zhì)儲(chǔ)層和有利區(qū)預(yù)測(cè)[D];東北石油大學(xué);2016年

2 俞玉芬;面向多核可伸縮的MapReduce庫(kù)的研究[D];中國(guó)科學(xué)技術(shù)大學(xué);2017年

3 陳慈;基于多核的Ramsey數(shù)算法研究[D];北京交通大學(xué);2015年

4 曹慧芳;多核下可擴(kuò)放確定性并行的MapReduce[D];中國(guó)科學(xué)技術(shù)大學(xué);2015年

，

本文編號(hào)：2417288