提出新型神经过程模型,高效处理大规模非网格气象数据。
Gridded Transformer Neural Processes for Large Unstructured Spatio-Temporal Data
- 引入网格伪标记机制,用高效注意力处理非网格观测数据。
- 在真实气象数据上优于多个基线模型,且计算效率不降。
- 适合需要大规模时空建模的气候与气象预测场景。
许多重要问题需建模大规模时空数据,如天气预报。近年来,基于Transformer的方法在天气预报中表现优异,但主要针对网格化数据,忽略了气象站等非网格观测数据。神经过程(NPs)特别是变压器神经过程(TNPs)适合此类任务,但现有TNPs难以扩展到先进气象与气候模型所用的数据量。这源于其缺乏高效的注意力机制。本文提出网格伪标记的TNP,采用专用编码器/解码器处理非网格观测,并使用包含网格伪标记的处理器,利用高效注意力机制。该方法在多种合成与真实世界回归任务中持续优于多个强基线,在大规模数据上保持竞争力计算效率。真实实验基于气象数据,证明该方法在天气建模流程中具备性能与计算优势。
原文摘要 · Abstract (English)
Many important problems require modelling large-scale spatio-temporal datasets, with one prevalent example being weather forecasting. Recently, transformer-based approaches have shown great promise in a range of weather forecasting problems. However, these have mostly focused on gridded data sources, neglecting the wealth of unstructured, off-the-grid data from observational measurements such as those at weather stations. A promising family of models suitable for such tasks are neural processes (NPs), notably the family of transformer neural processes (TNPs). Although TNPs have shown promise on small spatio-temporal datasets, they are unable to scale to the quantities of data used by state-of-the-art weather and climate models. This limitation stems from their lack of efficient attention mechanisms. We address this shortcoming through the introduction of gridded pseudo-token TNPs which employ specialised encoders and decoders to handle unstructured observations and utilise a processor containing gridded pseudo-tokens that leverage efficient attention mechanisms. Our method consistently outperforms a range of strong baselines on various synthetic and real-world regression tasks involving large-scale data, while maintaining competitive computational efficiency. The real-life experiments are performed on weather data, demonstrating the potential of our approach to bring performance and computational benefits when applied at scale in a weather modelling pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。