提出可扩展的结构化注意力模型，提升全球气象站预报精度。
S$^2$Transformer: Scalable Structured Transformers for Global Station Weather Forecasting
- 将空间图分块，分别建模局部与全局空间相关性
- 在多个尺度上逐步扩展子图，实现高效建模
- 适合需要高精度全球气象预测的科研与产业场景
全球气象站预报（GSWF）是能源、航空和农业等关键领域的研究重点。现有时间序列预测方法在大规模全球站点预报中常忽略或单向建模空间相关性，违背了全球天气系统的内在特性，制约了预测性能。为此，本文提出一种新型空间结构化注意力模块：将空间图划分为若干子图，通过子图内注意力学习局部空间相关性，并通过子图间注意力聚合节点表示以传递跨子图信息——同时考虑空间邻近性和全局相关性。基于该模块，构建多尺度时空预测模型S²Transformer，通过逐步扩大子图规模实现可扩展的结构化空间建模。实验表明，该模型在低计算开销下相较时间序列基线最高提升16.8%的性能。
原文摘要 · Abstract (English)
Global Station Weather Forecasting (GSWF) is a key meteorological research area, critical to energy, aviation, and agriculture. Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station forecasting. This contradicts the intrinsic nature underlying observations of the global weather system, limiting forecast performance. To address this, we propose a novel Spatial Structured Attention Block in this paper. It partitions the spatial graph into a set of subgraphs and instantiates Intra-subgraph Attention to learn local spatial correlation within each subgraph, and aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention -- considering both spatial proximity and global correlation. Building on this block, we develop a multiscale spatiotemporal forecasting model S$^2$Transformer by progressively expanding subgraph scales. The resulting model is both scalable and able to produce structured spatial correlation, and meanwhile, it is easy to implement. The experimental results show that it can achieve performance improvements up to 16.8% over time series forecasting baselines at low running costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。