提出在物理域计算注意力,让地球系统模型更准确可靠。
Field-Space Attention for Structure-Preserving Earth System Transformers
- 在物理空间而非隐空间计算注意力,保持场结构连续性。
- 在温度超分辨率任务中收敛更快,参数量少且更稳定。
- 适合需要物理一致性与可解释性的气候建模研究者。
精确且物理一致的地球系统动力学建模需要直接作用于连续地理场并保持其几何结构的机器学习架构。本文提出场空间注意力机制,使地球系统Transformer在物理域而非学习的隐空间中计算注意力。通过将所有中间表示保持为球面上的连续场,该架构实现可解释的内部状态,并便于施加科学约束。模型采用固定非学习的多尺度分解,学习输入场的结构保持形变,从而实现粗细尺度信息的协同融合,避免了标准单尺度视觉变压器常见的优化不稳定性。在HEALPix网格上的全球温度超分辨率任务中,场空间Transformer比传统视觉变压器和U-Net基线收敛更快、更稳定,且所需参数显著减少。网络全程显式保留场结构,使物理与统计先验可直接嵌入架构,提升数据驱动地球系统建模的保真度与可靠性。这些结果表明,场空间注意力是下一代地球系统预测与生成模型中紧凑、可解释且物理基础坚实的构建模块。
原文摘要 · Abstract (English)
Accurate and physically consistent modeling of Earth system dynamics requires machine-learning architectures that operate directly on continuous geophysical fields and preserve their underlying geometric structure. Here we introduce Field-Space attention, a mechanism for Earth system Transformers that computes attention in the physical domain rather than in a learned latent space. By maintaining all intermediate representations as continuous fields on the sphere, the architecture enables interpretable internal states and facilitates the enforcement of scientific constraints. The model employs a fixed, non-learned multiscale decomposition and learns structure-preserving deformations of the input field, allowing coherent integration of coarse and fine-scale information while avoiding the optimization instabilities characteristic of standard single-scale Vision Transformers. Applied to global temperature super-resolution on a HEALPix grid, Field-Space Transformers converge more rapidly and stably than conventional Vision Transformers and U-Net baselines, while requiring substantially fewer parameters. The explicit preservation of field structure throughout the network allows physical and statistical priors to be embedded directly into the architecture, yielding improved fidelity and reliability in data-driven Earth system modeling. These results position Field-Space Attention as a compact, interpretable, and physically grounded building block for next-generation Earth system prediction and generative modeling frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。