用函数图像测度构建新框架,让Transformer能通用逼近函数空间间算子。
Function graph transformers universally approximate operators between function spaces

- 将函数转为图像测度,用测度视角重写Transformer结构。
- 证明其可逼近多种非线性算子,支持不同离散化和输出域。
- 适合研究算子学习、连续建模的学者,强化了位置编码与正则作用。
本文研究用Transformer逼近函数空间间的非线性算子。方法是将函数$h$表示为其图像测度$γ_h$,通过有限采样点集$\{(x_j,h(x_j))\}_{j=1}^N$实现经验近似。该框架自然建模离散化细化过程,为算子学习提供统一视角。文中提出函数图像Transformer——一类保持图像结构的测度变换器,输出仍为单值函数。关键结果表明,这类映射可通过有限层自注意力与点对点MLP组合逼近,实现对广泛非线性算子的通用逼近。相比现有理论,该框架还能处理带负阶Sobolev正则化的输入及不同输出域的查询点。整体提供了基于连续观点的数学工具,厘清位置编码、图结构、正则化的作用,并确保离散化一致性。
原文摘要 · Abstract (English)
We study the approximation of nonlinear operators between function spaces by transformers. Our approach is to lift functions to measures supported on their graphs and leverage a recently introduced measure-theoretic view of transformers. A function $h$ is represented by its graph measure $γ_h$, with finite tokens $\{(x_j,h(x_j))\}_{j=1}^N$ being its empirical approximations. We show that this framework elegantly models discretization refinement via convergence of measures and provides a natural setting for operator learning. Within this framework, we introduce function graph transformers, a graph-preserving subclass of measure-theoretic transformers that maps graph measures to graph measures, which is to say that outputs remain single-valued functions. Crucially, this additional structure does not reduce generality: we prove that the resulting graph-preserving maps can be approximated by finite compositions of standard softmax self-attention layers and pointwise MLPs, yielding universal approximation results for broad classes of nonlinear operators. Unlike existing theoretical approaches to operator learning with transformers, the measure-theoretic framework also accommodates regularized negative-order Sobolev inputs for which discretization invariance is particularly challenging, as well as query points on different output domains. Overall, function graph transformers provide a continuum viewpoint and mathematical toolkit for transformer-based operator learning, clarifying the roles of positional encodings, graph structure, regularization, and ensuring consistency across discretizations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。