用域分解方法让Transformer高效处理高分辨率偏微分方程
Mondrian: Transformer Operators via Domain Decomposition
- 将域划分为不重叠子域，注意力仅在子域函数间计算
- 在Allen-Cahn和Navier-Stokes上实现无重训练的分辨率扩展
- 适合需高分辨率、多尺度物理建模的研究者
算子学习通过学习函数空间间的映射实现偏微分方程（PDE）的数据驱动建模。然而，由于注意力机制存在二次复杂度且与离散化耦合，基于Transformer的算子模型在高分辨率、多尺度域上的扩展仍面临挑战。本文提出 extbf{Mondrian}，一种基于域分解的Transformer算子：将域划分为非重叠子域，在子域内限制函数并计算注意力。利用域分解思想，该方法将注意力与离散化解耦。每个子域内使用表达性强的神经算子替代标准层，跨子域注意力通过函数的softmax内积计算。该框架自然支持层次化窗口注意力与邻域注意力，兼顾局部与全局交互。Mondrian在Allen-Cahn和Navier-Stokes PDE上表现优异，实现无需重训练的分辨率扩展。结果表明，域分解注意力为可扩展、通用的神经算子提供了新路径。
原文摘要 · Abstract (English)
Operator learning enables data-driven modeling of partial differential equations (PDEs) by learning mappings between function spaces. However, scaling transformer-based operator models to high-resolution, multiscale domains remains a challenge due to the quadratic cost of attention and its coupling to discretization. We introduce \textbf{Mondrian}, transformer operators that decompose a domain into non-overlapping subdomains and apply attention over sequences of subdomain-restricted functions. Leveraging principles from domain decomposition, Mondrian decouples attention from discretization. Within each subdomain, it replaces standard layers with expressive neural operators, and attention across subdomains is computed via softmax-based inner products over functions. The formulation naturally extends to hierarchical windowed and neighborhood attention, supporting both local and global interactions. Mondrian achieves strong performance on Allen-Cahn and Navier-Stokes PDEs, demonstrating resolution scaling without retraining. These results highlight the promise of domain-decomposed attention for scalable and general-purpose neural operators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。