构建可视化大数据系统,提升跨区域气象预测精度。
A visual big data system for the prediction of weather-related variables: Jordan-Spain case study
- 融合多源气象数据,支持时空聚合与缺失值补全预测。
- 温度与降水预测误差均方根达0.00013,方向一致性近0.84。
- 适合气象研究者与气候数据分析人员使用。
气象学产生海量数据,主要来自气象站传感器,包含温度、降雨量等变量。这些数据具有高维度、高缺失率及强相关性等特点。本文提出一个可视化大数据系统,用于处理大规模气象数据,支持对温度和降雨量的预测分析。系统通过本地NoSQL数据库整合开放数据,在不同时空粒度上进行数据融合,并采用单变量、多变量建模以及邻站数据迁移学习来应对高缺失场景。评估显示,系统整体归一化均方误差为0.00013,方向对称性接近0.84。专家评审中除图形设计外,其余方面评分均在3分以上(满分5分)。初步结果验证了系统的有效性,具备进一步研究价值。
原文摘要 · Abstract (English)
The Meteorology is a field where huge amounts of data are generated, mainly collected by sensors at weather stations, where different variables can be measured. Those data have some particularities such as high volume and dimensionality, the frequent existence of missing values in some stations, and the high correlation between collected variables. In this regard, it is crucial to make use of Big Data and Data Mining techniques to deal with those data and extract useful knowledge from them that can be used, for instance, to predict weather phenomena. In this paper, we propose a visual big data system that is designed to deal with high amounts of weather-related data and lets the user analyze those data to perform predictive tasks over the considered variables (temperature and rainfall). The proposed system collects open data and loads them onto a local NoSQL database fusing them at different levels of temporal and spatial aggregation in order to perform a predictive analysis using univariate and multivariate approaches as well as forecasting based on training data from neighbor stations in cases with high rates of missing values. The system has been assessed in terms of usability and predictive performance, obtaining an overall normalized mean squared error value of 0.00013, and an overall directional symmetry value of nearly 0.84. Our system has been rated positively by a group of experts in the area (all aspects of the system except graphic desing were rated 3 or above in a 1-5 scale). The promising preliminary results obtained demonstrate the validity of our system and invite us to keep working on this area.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。