用注意力机制提升空气质量预测精度，最高降误差22%
A HEART for the environment: Transformer-Based Spatiotemporal Modeling for Air Quality Prediction
- 在卷积网络前加自注意力模块，融合历史数据与外部信息
- 平均误差降低7.5%，最高降幅达22%（针对不同污染物）
- 方法可推广至其他时间序列预测，尤其适合固定长度数据
精准可靠的空气污染预测对环境管理与政策制定至关重要。llull-environment 是一个受马德里和巴利亚多利德现有系统启发的复杂且可扩展的污染预测系统，包含编码器-解码器卷积神经网络，用于预测四种主要污染物（NO₂、O₃、PM₁₀、PM₂.₅）的平均浓度，输入包括历史数据、外部预报及其他上下文特征。本文研究通过引入注意力机制增强该神经网络以提升预测精度。所提注意力机制在输入特征张量传入原有均值预测模型前进行预处理。最终模型为多种架构与思想的结合，可称为“混合增强自回归变换器”（HEART）。通过对比不同注意力结构与无此机制系统的均方误差（MSE），评估其有效性。结果显示，在测试的城市和污染物中，平均误差下降7.5%，最高降幅达22%。不同污染物的表现差异显著，反映其生成与消散过程的不同特性。研究结论不仅适用于优化空气质量预测模型，也可广泛应用于（固定长度）时间序列预测任务。
原文摘要 · Abstract (English)
Accurate and reliable air pollution forecasting is crucial for effective environmental management and policy-making. llull-environment is a sophisticated and scalable forecasting system for air pollution, inspired by previous models currently operational in Madrid and Valladolid (Spain). It contains (among other key components) an encoder-decoder convolutional neural network to forecast mean pollution levels for four key pollutants (NO$_2$, O$_3$, PM$_{10}$, PM$_{2.5}$) using historical data, external forecasts, and other contextual features. This paper investigates the augmentation of this neural network with an attention mechanism to improve predictive accuracy. The proposed attention mechanism pre-processes tensors containing the input features before passing them to the existing mean forecasting model. The resulting model is a combination of several architectures and ideas and can be described as a "Hybrid Enhanced Autoregressive Transformer", or HEART. The effectiveness of the approach is evaluated by comparing the mean square error (MSE) across different attention layouts against the system without such a mechanism. We observe a significant reduction in MSE of up to 22%, with an average of 7.5% across tested cities and pollutants. The performance of a given attention mechanism turns out to depend on the pollutant, highlighting the differences in their creation and dissipation processes. Our findings are not restricted to optimizing air quality prediction models, but are applicable generally to (fixed length) time series forecasting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。