用状态空间模型提升神经算子，更准更快求解偏微分方程
Mamba Neural Operator: Who Wins? Transformers vs. State-Space Models for PDEs
- 将结构化状态空间模型与神经算子结合，统一建模框架
- 在多个PDE任务上精度显著超越Transformer，长程依赖捕捉更强
- 适合需要高精度连续动态建模的科学计算场景
偏微分方程（PDE）广泛用于建模复杂物理系统，但高效求解仍是重大挑战。近年来，变压器（Transformers）因其捕捉复杂依赖关系的能力成为主流架构，但在表示连续动态和长程相互作用方面存在不足。为此，我们提出一种新框架——马巴神经算子（Mamba Neural Operator, MNO），建立结构化状态空间模型（SSMs）与神经算子之间的正式理论联系，提供可适配多种架构的统一结构。通过利用SSMs的结构设计，MNO比传统Transformer更有效地捕捉长程依赖与连续动态。大量实验表明，MNO显著提升了神经算子的表达能力和精度，不仅作为补充，更成为优于现有方法的框架，弥合了高效表示与精确解逼近之间的差距。代码已开源：https://github.com/Math-ML-X/Mamba-Neural-Operator
原文摘要 · Abstract (English)
Partial differential equations (PDEs) are widely used to model complex physical systems, but solving them efficiently remains a significant challenge. Recently, Transformers have emerged as the preferred architecture for PDEs due to their ability to capture intricate dependencies. However, they struggle with representing continuous dynamics and long-range interactions. To overcome these limitations, we introduce the Mamba Neural Operator (MNO), a novel framework that enhances neural operator-based techniques for solving PDEs. MNO establishes a formal theoretical connection between structured state-space models (SSMs) and neural operators, offering a unified structure that can adapt to diverse architectures, including Transformer-based models. By leveraging the structured design of SSMs, MNO captures long-range dependencies and continuous dynamics more effectively than traditional Transformers. Through extensive analysis, we show that MNO significantly boosts the expressive power and accuracy of neural operators, making it not just a complement but a superior framework for PDE-related tasks, bridging the gap between efficient representation and accurate solution approximation. Our code is available on https://github.com/Math-ML-X/Mamba-Neural-Operator
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。