自然资源分区矢量数据的快速并行统计方法Fast parallel statistical method for partitioned vector data of natural resources
亢晓琛,董春,栗斌,杜林丹,钱兴隆
摘要(Abstract):
针对自然资源数据统计计算所面临的高耗时问题,该文提出了一种支持大规模分区矢量数据的快速并行统计方法。基于该方法构建的高性能计算系统可以解决分布式数据组织、集群计算和统计模型管理3方面问题,从而为利用高性能计算设施提升统计计算效率提供一种有效手段。实验以表面面积和地表覆盖转移矩阵为例,在集群环境下完成全国陆域全要素数据计算测试,性能提升分别达到113.95倍与65.56倍,综合验证了该文方法的性能优势。
关键词(KeyWords): 自然资源;分区矢量数据;并行计算;表面面积;转移矩阵
基金项目(Foundation): 国家自然科学基金项目(41701461);; 中国测绘科学研究院基本科研业务费项目(AR2001)
作者(Author): 亢晓琛,董春,栗斌,杜林丹,钱兴隆
DOI: 10.16251/j.cnki.1009-2307.2022.04.026
参考文献(References):
- [1] 李清泉,李德仁.大数据GIS[J].武汉大学学报(信息科学版),2014,39(6):641-644.(LI Qingquan,LI Deren.Big data GIS[J].Geomatics and Information Science of Wuhan University,2014,39(6):641-644.)
- [2] 刘纪平,董春,亢晓琛,等.大数据时代的地理国情统计分析[J].武汉大学学报(信息科学版),2019,44(1):68-76.(LIU Jiping,DONG Chun,KANG Xiaochen,et al.National geographical conditions statistical analysis in the era of big data[J].Geomatics and Information Science of Wuhan University,2019,44(1):68-76.)
- [3] 张兵.遥感大数据时代与智能信息提取[J].武汉大学学报(信息科学版),2018,43(12):1861-1871.(ZHANG Bing.Remotely sensed big data era and intelligent information extraction[J].Geomatics and Information Science of Wuhan University,2018,43(12):1861-1871.)
- [4] KANG Xiaochen,LIU Jiping,DONG Chun,et al.Using high-performance computing to address the challenge of land use/land cover change analysis on spatial big data[J].ISPRS International Journal of Geo-Information,2018,7(7):273.
- [5] 自然资源部.自然资源部关于印发《自然资源调查监测体系构建总体方案》的通知[EB/OL].(2020-01-17)[2021-09-14].http://dkj.ah.gov.cn/public/7031/40401561.html.(Ministry of Natural Resources.Ministry of Natural Resources released notification on《the overall scheme of natural resources survey and monitoring system construction》[EB/OL].(2020-01-17)[2021-09-14].http://dkj.ah.gov.cn/public/7031/40401561.html.)
- [6] MACIEL A M,CAMARA G,VINHAS L,et al.A spatiotemporal calculus for reasoning about land-use trajectories[J].International Journal of Geographical Information Science,2019,33(1):176-192.
- [7] 姚晓闯.矢量大数据管理关键技术研究[D].北京:中国农业大学,2017.(YAO Xiaochuang.Research on key te cchnologies of vector big data management[D].Beijing:China Agricultural University,2017.)
- [8] 裴韬,刘亚溪,郭思慧,等.地理大数据挖掘的本质[J].地理学报,2019,74(3):586-598.(PEI Tao,LIU Yaxi,GUO Sihui,et al.Principle of big geodata mining[J].Acta Geographica Sinica,2019,74(3):586-598.)
- [9] 周成虎,程维明,钱金凯,等.中国陆地1∶100万数字地貌分类体系研究[J].地球信息科学学报,2009,11(6):707-724.(ZHOU Chenghu,CHENG Weiming,QIAN Jinkai,et al.Research on the classification system of digital land geomorphology of 1∶1 000 000 in China[J].Journal of Geo-Information Science,2009,11(6):707-724.)
- [10] 熊礼阳,汤国安,杨昕,等.面向地貌学本源的数字地形分析研究进展与展望[J].地理学报,2021,76(3):595-611.(XIONG Liyang,TANG Guoan,YANG Xin,et al.Geomorphology-oriented digital terrain analysis:Progress and perspectives[J].Acta Geographica Sinica,2021,76(3):595-611.)
- [11] 何红艳,郭志华,肖文发.降水空间插值技术的研究进展[J].生态学杂志,2005,24(10):1187-1191.(HE Hongyan,GUO Zhihua,XIAO Wenfa.Review on spatial interpolation techniques of rainfall[J].Chinese Journal of Ecology,2005,24(10):1187-1191.)
- [12] 张成成,桂德竹.自然资源管理中权属界线与空间管控界线测绘精度的问题分析[J].测绘通报,2020(12):106-109.(ZHANG Chengcheng,GUI Dezhu.Analysis of some problems of surveying and mapping accuracy of ownership boundary and spatial control boundary in natural resource management[J].Bulletin of Surveying and Mapping,2020(12):106-109.)
- [13] 陈军,赵仁亮.GIS空间关系的基本问题与研究进展[J].测绘学报,1999,28(2):4-11.(CHEN Jun,ZHAO Renliang.Spatial relations in GIS:A survey on its key issues and research progress[J].Acta Geodaetica et Cartographic Sinica,1999,28(2):4-11.)
- [14] KANG Xiaochen,LIN Xiangguo.Graph-based divide and conquer method for parallelizing spatial operations on vector data[J].Remote Sensing,2014,6(10):10107-10130.
- [15] AJI A,WANG Fusheng,VO H,et al.Hadoop-GIS:A high performance spatial data warehousing system over MapReduce[EB/OL].[2021-09-14].https://pubmed.ncbi.nlm.nih.gov/24187650/.
- [16] 于雷易,边馥苓,万丰.一种多边形交、并、差运算的有效算法[J].武汉大学学报(信息科学版),2003,28(5):615-618.(YU Leiyi,BIAN Fuling,WAN Feng.An efficient algorithm for intersection,union and difference between polygons[J].Geomatics and Information Science of Wuhan University,2003,28(5):615-618.)
- [17] 桑应宾.基于K近邻的分类算法研究[D].重庆:重庆大学,2009.(SANG Yingbin.Research of classification algorithm based on K nearest neighbor[D].Chongqing:Chongqing University,2009.)
- [18] 黄继先.基于R-树的空间数据库查询技术研究[D].长沙:中南大学,2005.(HUANG Jixian.The query techniques of spatial database on R-tree[D].Changsha:Central South University,2005.)
- [19] 乐阳,龚健雅.Dijkstra最短路径算法的一种高效率实现[J].武汉测绘科技大学学报,1999,24(3):209-212.(YUE Yang,GONG Jianya.An efficient implementation of shortest path algorithm based on Dijkstra algorithm[J].Journal of Wuhan Technical University of Durveying and Mapping,1999,24(3):209-212.)
- [20] 邱强,秦承志,朱效民,等.全空间下并行矢量空间分析研究综述与展望[J].地球信息科学学报,2017,19(9):1217-1227.(QIU Qiang,QIN Chengzhi,ZHU Xiaomin,et al.Overview and prospect on spatial analysis of parallel vectors in pan-spatial concept[J].Journal of Geo-information Science,2017,19(9):1217-1227.)
- [21] 李京,孙颖博,刘智深,等.模型库管理系统的设计和实现[J].软件学报,1998,9(8):54-59.(LI Jing,SUN Yingbo,LIU Zhishen,et al.Design and implementation of a model base management system[J].Journal of Software,1998,9(8):54-59.)
- [22] 宫辉力,李京,陈秀万,等.地理信息系统的模型库研究[J].地学前缘,2000(S2):17-22.(GONG Huili,LI Jing,CHEN Xiuwan,et al.Study on model base system of GIS[J].Earth Science Frontiers,2000(S2):17-22.)
- [23] ISO 19119-2016 Geographic information -services [S/OL].(2016-05--29)[ 2021-07-23].https://www.doc88.com/p-1465284078602.html?r=1:2016.
- [24] 于海龙,邬伦,刘瑜,等.基于Web Services的GIS与应用模型集成研究[J].测绘学报,2006,35(2):153-159.(YU Hailong,WU Lun,LIU Yu,et al.A study of integration between GIS and GIS-based model based on Web Services[J].Acta Geodaetica et Cartographica Sinica,2006,35(2):153-159.)