arXiv:2605.21507physics.ao-phcs.AI2026-05

解决韩国能见度预测中的数据不平衡与分布偏移问题

Visibility nowcasting in South Korea: a machine learning approach to class imbalance and distribution shift

论文配图:Visibility nowcasting in South Korea: a machine learning approach to class imbalance and distribution shift
图 1 · 摘自论文原文
  • 用SMOTENC和CTGAN处理低频低能见度事件的数据不足
  • 模型在2021年测试集上性能显著下降,因数据分布变化
  • 通过SHAP和Wasserstein距离量化了关键特征的分布偏移

大气能见度对交通安全和空气质量管控至关重要,但受气象条件与污染物复杂交互影响,且低能见度事件罕见,预测难度大。本研究针对韩国六大城市提出一种机器学习框架用于能见度短时预报。为应对2018-2020年训练数据中的类别不平衡问题,采用SMOTENC与条件表格式生成对抗网络(CTGAN)进行数据增强。随后构建集成模型,结合机器学习与深度学习方法,并在2021年测试集上评估性能。结果发现,测试集表现显著劣于交叉验证阶段,该退化归因于训练与测试期间数据分布的变化,通过SHAP分析识别出关键特征后,以Wasserstein距离定量验证了分布偏移。总体而言,本研究提出的方法同时应对数据不平衡与时间分布偏移双重挑战,强调在时序数据中实施短时预报模型时需考虑外部环境因素的动态演化。

原文摘要 · Abstract (English)

Atmospheric visibility is a critical variable for transportation safety and air quality management, however, accurate prediction remains challenging due to the complex interactions between meteorological conditions and air pollutants, as well as the rarity of low-visibility events. This study introduces a machine learning framework to nowcast visibility in six major South Korean cities. To handle the imbalance in the 2018-2020 training data, we applied the Synthetic Minority Over-sampling Technique with Nominal and Continuous (SMOTENC) and Conditional Tabular Generative Adversarial Network (CTGAN). An ensemble approach combining machine learning and deep learning models was then used and evaluated on a 2021 test dataset. The results revealed a marked decline in predictive performance in the test set compared to the cross-validation phase. This degradation was attributed to a distributional shift between training and testing periods, which was quantitatively confirmed by measuring the Wasserstein distance of the most influential feature identified by SHAP analysis. In general, this study presents a methodology that aims to simultaneously address the dual challenges of data imbalance and temporal distributional shifts, and emphasizes the necessity of accounting for evolving external environmental factors when implementing nowcasting models on time-series data.

能见度预测数据不平衡分布偏移时序建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。