提升降水预报稳定性,通过调控注意力能量减少预测波动。
Stable Attention Response for Reliable Precipitation Nowcasting

- 提出头部级注意力能量正则化,降低跨样本响应波动。
- 在SEVIR和MeteoNet数据集上实现当前最佳预报精度。
- 适用于单模态与多模态架构,提升模型可靠性。
降水短时预报因大气动力学的高度局部性、快速演化性和异质性而极具挑战。尽管近年方法越来越多地采用基于注意力的架构,但主要关注增强表征能力和预测性能,却忽视了注意力响应在不同样本间的稳定性。本文发现,跨样本注意力能量的不稳定性是导致预报不可靠的重要且此前未被充分研究的来源。实证表明,预测误差较大的情况往往伴随注意力头与层间能量方差更大。理论上,我们证明了跨样本变异性可通过自注意力传播,并放大预测误差的下界。基于此,提出HARECast——一种针对降水预报的头部级注意力响应能量调控框架。该框架显式建模并利用组内正则化目标稳定各注意力头的能量响应,降低跨样本波动。该方法具有通用性,可应用于单模态与多模态预报架构。我们在标准预报流程中集成重建分支与基于扩散的预测器,在常用基准数据集SEVIR和MeteoNet上验证,结果表明HARECast达到当前最优性能。
原文摘要 · Abstract (English)
Precipitation nowcasting remains challenging due to the highly localized, rapidly evolving, and heterogeneous nature of atmospheric dynamics. Although recent methods increasingly adopt attention-based architectures in both unimodal and multimodal settings, they mainly emphasize stronger representation learning and prediction capacity, while paying less attention to the stability of attention responses across samples. In this work, we show that cross-sample instability of attention-response energy is an important and previously underexplored source of forecasting unreliability. Empirically, inaccurate forecasts are associated with larger attention-response energy variance across heads and layers. Theoretically, we show that cross-sample variability can propagate through self-attention, and enlarge a lower bound on prediction error. Based on this insight, we propose HARECast, a Head-wise Attention Response Energy-regulated framework for precipitation nowcasting. HARECast explicitly models head-wise attention-response energy and stabilizes it through a group-wise regularization objective that reduces cross-sample fluctuations. The proposed formulation is generic and applicable to both unimodal and multimodal nowcasting architectures. We instantiate HARECast in a standard forecasting pipeline with reconstruction branches and a diffusion-based predictor, and evaluate it on commonly used benchmarks--SEVIR and MeteoNet. Experimental results demonstrate that HARECast achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。