用因果注意力建模降水预报中的时空因果关系,提升长期依赖捕捉能力。
Precipitation Nowcasting Using Diffusion Transformer with Causal Attention
- 引入因果注意力机制,建立条件与预报结果间的时空查询关系。
- 在两个数据集上预测强降水的临界成功指数提升15%和8%。
- 适合需要高精度短期降水预报与可解释性的气象应用。
短时降水预报因难以捕捉长期时空依赖而面临挑战。现有深度学习方法在建立条件与预报结果间有效依赖关系方面表现不足,且缺乏可解释性。为此,我们提出基于因果注意力的扩散变压器(Diffusion Transformer with Causal Attention, DTCA)模型。该模型结合Transformer架构与因果注意力机制,建立条件信息(原因)与预报结果(结果)之间的时空查询关系,有效捕捉长程依赖,使预报结果在时空范围内保持强因果关联。我们探索了四种时空信息交互方式,发现全局时空标签交互性能最佳。此外,引入通道转批次(Channel-To-Batch shift)操作以增强复杂降雨动态的表征能力。在两个数据集上的实验表明,相较于领先的基于U-Net的方法,本方法在预测强降水时的临界成功指数(CSI)分别提升约15%和8%,达到当前最优性能。
原文摘要 · Abstract (English)
Short-term precipitation forecasting remains challenging due to the difficulty in capturing long-term spatiotemporal dependencies. Current deep learning methods fall short in establishing effective dependencies between conditions and forecast results, while also lacking interpretability. To address this issue, we propose a Precipitation Nowcasting Using Diffusion Transformer with Causal Attention model. Our model leverages Transformer and combines causal attention mechanisms to establish spatiotemporal queries between conditional information (causes) and forecast results (results). This design enables the model to effectively capture long-term dependencies, allowing forecast results to maintain strong causal relationships with input conditions over a wide range of time and space. We explore four variants of spatiotemporal information interactions for DTCA, demonstrating that global spatiotemporal labeling interactions yield the best performance. In addition, we introduce a Channel-To-Batch shift operation to further enhance the model's ability to represent complex rainfall dynamics. We conducted experiments on two datasets. Compared to state-of-the-art U-Net-based methods, our approach improved the CSI (Critical Success Index) for predicting heavy precipitation by approximately 15% and 8% respectively, achieving state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。