将注意力机制重构为函数对应关系，实现对连续场的全局建模。
Functional Attention: From Pairwise Affinities to Functional Correspondences
- 用自适应基函数间的结构化线性映射替代传统注意力
- 在求解PDE、3D分割等任务中达到顶尖性能且对离散化不敏感
- 适合需要处理连续函数空间的机器学习任务
学习无穷维函数空间间的映射，即算子学习，在众多机器学习应用中至关重要。尽管基于Transformer的算子方法流行，但通常依赖逐项注意力，将连续场视为离散标记，常忽略全局函数结构。本文提出 extit{Functional Attention}，将注意力重新解释为自适应基函数间的函数对应。受几何函数映射启发，该方法用结构化线性算子取代softmax亲和度。这生成了紧凑、可泛化、分辨率不变的表示，显式捕捉全局依赖。实验表明， extit{Functional Attention}在求解偏微分方程（PDE）、3D分割和回归等算子学习任务中可达到顶尖性能，且对不同离散化保持鲁棒。项目页面见https://github.com/xjffff/FUNCATTN。
原文摘要 · Abstract (English)
Learning mappings between infinite-dimensional function spaces, or operator learning, is essential for many machine learning applications. Although transformer-based operators are popular, they often rely on token-wise attention. These methods treat continuous fields as discrete tokens and usually ignore the global functional structure. We introduce \emph{Functional Attention}, which reinterprets attention as a functional correspondence between adaptive bases. Inspired by geometric functional maps, our method replaces softmax affinities with structured linear operators. This yields a compact, generalizable, resolution-invariant representation that explicitly captures global dependencies. Experiments demonstrate that \emph{Functional Attention} can match state-of-the-art performance in many operator learning tasks, including solving PDEs, 3D segmentation, and regression, while remaining robust to varying discretizations. Project page is available at https://github.com/xjffff/FUNCATTN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。