基于MapReduce的CME参数识别模型并行计算技术

杨世通; 蔡燕霞; 鲁国瑞; 王晶晶

doi:10.11728/cjss2020.02.169

基于MapReduce的CME参数识别模型并行计算技术

doi: 10.11728/cjss2020.02.169

1. 中国科学院国家空间科学中心北京 100190;
2. 中国科学院大学北京 100049

基金项目:

国家自然科学基金项目（41604149）和北京市科技重大专项（Z181100002918004）共同资助

详细信息

作者简介:
杨世通,E-mail:yangst0101@163.com

中图分类号: P353
计量
- 文章访问数: 758
- HTML全文浏览量: 45
- PDF下载量: 100
- 被引次数: 0
出版历程
- 收稿日期: 2019-03-12
- 修回日期: 2019-11-03
- 刊出日期: 2020-03-15

Parallel Computing Technology for CME Parameter Detection Model Based on MapReduce

1 National Space Science Center, Chinese Academy of Sciences, Beijing 100190;
2 University of Chinese Academy of Sciences, Beijing 100049

摘要

摘要: 日冕物质抛射（Coronal Mass Ejection，CME）参数识别模型是太阳风预报过程的重要组成部分.在空间环境预报业务中，为提高太阳风预报的准确率，需要提高CME参数识别的精度.模型以计算任务串行的方式运行，运算效率低导致模型运算时间长，不能满足这种需求.CME参数识别模型的物理运算过程相互不独立，其在单节点上的运行方式不能满足并行化要求.基于MapReduce的并行计算框架，改进了CME参数识别模型的计算流程，提出CDMR（CME detection under MapReduce）方法，实现了CME参数识别模型的并行计算，并对比分析CME参数识别模型在串行计算和MapReduce并行计算下的运行时间，提高了模型的识别精度和计算效率.
- CME参数识别模型 /
- MapReduce /
- 并行计算效率
Abstract: Space environment prediction model is an important part of space environment business. Coronal Mass Ejection (CME) is the source of many space events and near-Earth space environment disturbances. The CME parameter detection model is an important part of the solar wind forecasting process. In order to improve the accuracy of solar wind forecasting in space environment forecasting, it is necessary to improve the accuracy of CME parameter detection. However, the model runs in serial mode with low calculating efficiency, which leads to long operation time of the model and can not meet the requirement. Based on the parallel computing framework of MapReduce, according to the characteristics of CME parameter detection model, the calculation flow of CME parameter detection model is improved. A CDMR (CME Detection under MapReduce) method is presented, which can realize the parallel computing of CME parameter detection model. Moreover, the running time of the CME parameter detection model between serial computing and MapReduce parallel computing is compared. The experimental results show that the running time is reduced by using MapReduce parallel computing, and the detection accuracy and calculation efficiency of the model are improved.
- CME parameters detection model /
- MapReduce /
- Parallel calculating efficiency

HTML全文

参考文献(17)

[1]	WANG Jingjing, LUO Bingxian, LIU Siqing, et al. Analysis of CME events in 2010 combined with in-situ and STEREO/HI observations[J]. Chin. J. Geophys., 2013, 56(03):746-757(王晶晶, 罗冰显, 刘四清, 等. 结合实地观测和STEREO/HI图像观测分析2010年CME事件[J]. 地球物理学报, 2013, 56(03):746-757)
[2]	ZHANG Yingnan, GU Naijie, PENG Jianzhang, et al. A kernel level session-persistence method for multi-process load balancing[J]. Comput. Eng., 2014, 40(3):76-81(张颖楠, 顾乃杰, 彭建章, 等. 一种内核级多进程负载均衡会话保持方法[J]. 计算机工程, 2014, 40(3):76-81)
[3]	DEAN J, GHEMAWAT S. MapReduce:simplified data processing on large clusters[C]//Proceedings of Operating Systems Design and Implementation. San Francisco:CA, 2004:137-150
[4]	GHEMAWAT S, GOBIOFF H, LEUNG S. The google file system[J]. Sacm Sigops Oper. Syst. Rev., 2003, 37(5):29-43
[5]	ZHUANG Bin, WANG Yuming, SHEN Chenglong, et al. The significance of the influence of the CME deflection in interplanetary space on the CME arrival at Earth[J]. Astrophys. J., 2017, 845(2):117
[6]	WANG Jingjing, AO Xianzhi, WANG Yuming, et al. An operational solar wind prediction system transitioning fundamental science to operations[J]. J. Space Weather Space Clim., 2018, 8(A39).DOI: http://doi.org/10.1051/swsc/2018025
[7]	SHEELEY N R, WALTERS J H, WANG Y M, et al. Continuous tracking of coronal outflows:two kinds of coronal mass ejctions[J]. J. Geophys. Res., 1999, 104:24739-24767
[8]	DAVIES J A, HARRISON R A, ROUILLARD A P, et al. A synoptic view of solar transient evolution in the inner heliosphere using the Heliospheric Imagers on STEREO[J]. Geophys. Res. Lett., 2009, 36(2):L02102
[9]	CHEN Aiping. Research on Parallelization Analysis and Application of Clustering Algorithm Based on Hadoop[D]. Chengdu:University of Electronic Science and Technology of China, 2015
[10]	XIA Dawen. MapReduce-based Methodologies of Mobile Trajectory Big Data Mining and Its Application[D]. Chongqing:Southwest University, 2016
[11]	ZHANG Wenjie, JIANG Liehui. Parallel computation algorithm for big data clustering based onMapReduce[OL].[2018-12-1]. https://doi.org/10.19734/j.issn.1001-3695.2018.05.0496(张文杰, 蒋烈辉. 一种基于MapReduce并行化计算的大数据聚类算法[OL].[2018-12-1]. https://doi.org/10.19734/j.issn.1001-3695.2018.05.0496)
[12]	WU Xindong, JI Shengwei. Comparative Study on MapReduce and Spark for big data analytics[J]. J. Software, 2018, 29(6):1770-1791(吴信东, 嵇圣砛. MapReduce与Spark用于大数据分析之比较[J]. 软件学报, 2018, 29(6):1770-1791)
[13]	DOMINGO V, FLECK B, OOLAND A I. The SOHO mission:an overview[J]. Sol. Phys., 1995, 162(1/2):1-37
[14]	BRUECKNER G E, HOWARD R A, KOOMEN M J, et al. The large angle spectroscopic coronagraph (LASCO)[J]. Sol. Phys., 1995, 162(1/2):357-402
[15]	THOMPSON W T. Coordinate systems for solar image data[J]. Astron. Astrophys., 2006, 449:791-803
[16]	WANG Jingjing, LUO Bingxian, LIU siqing,et al. Modification and study of Self-Similar Expansion(SSE) model[J]. Chin. J. Geophys., 2013, 56(9):2871-2884(王晶晶, 罗冰显, 刘四清, 等. 对自相似扩展(SSE)模型的改进和研究[J]. 地球物理学报, 2013, 56(9):2871-2884)
[17]	LIU J, ZHU A, QIN C. Estimation of theoretical maximum speedup ratio for parallel computing of grid-based distributed hydrological models[J]. Comput. Geosci., 2013, 60(10):58-62