提出新方法在流形上近似双变量函数，提升核函数建模精度。
Manifold Random Features
- 基于流形离散化与图随机特征构建连续场近似机制
- 提供正定有界特征，实现低方差高精度逼近
- 可复现线性注意力Transformer中的高斯核近似，简化计算
我们提出一种新的随机特征生成范式——流形随机特征（Manifold Random Features, MRFs），用于近似定义在一般流形上的双变量函数（特别是核函数）。该方法结合流形的离散化与近期提出的图随机特征（GRFs）技术，学习流形上的连续场，从而获得原本难以解析推导的连续逼近机制。MRFs 生成正定且有界的特征，这是实现高精度、低方差近似的必要条件。我们揭示了定义在离散图对象上的 GRFs 与用于常规核函数的连续随机特征之间的深层渐近联系。作为副产品，我们的方法重新发现并简化了近期用于改进线性注意力 Transformer 的高斯核近似机制，仅通过图上的简单随机游走即可实现，避免了原始复杂的数学推导。我们提供了严谨的理论分析，并通过全面的实验验证了方法的有效性。
原文摘要 · Abstract (English)
We present a new paradigm for creating random features to approximate bi-variate functions (in particular, kernels) defined on general manifolds. This new mechanism of Manifold Random Features (MRFs) leverages discretization of the manifold and the recently introduced technique of Graph Random Features (GRFs) to learn continuous fields on manifolds. Those fields are used to find continuous approximation mechanisms that otherwise, in general scenarios, cannot be derived analytically. MRFs provide positive and bounded features, a key property for accurate, low-variance approximation. We show deep asymptotic connection between GRFs, defined on discrete graph objects, and continuous random features used for regular kernels. As a by-product of our method, we re-discover recently introduced mechanism of Gaussian kernel approximation applied in particular to improve linear-attention Transformers, considering simple random walks on graphs and by-passing original complex mathematical computations. We complement our algorithm with a rigorous theoretical analysis and verify in thorough experimental studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。