提出线性复杂度的注意力机制，高效建模原子间远距离相互作用。
Euclidean Fast Attention -- Machine Learning Global Atomic Representations at Linear Cost
- 基于欧氏旋转位置编码，实现空间对称性保持的线性注意力
- 在化学力场中准确捕捉传统方法失败的长程相互作用
- 适合需要高效建模空间关系的分子模拟与材料设计任务
长程相关性在众多机器学习任务中至关重要，尤其在欧氏空间数据中，远距离组分的相对位置与取向常决定预测精度。自注意力虽能有效捕捉全局效应，但其二次复杂度带来显著计算瓶颈。该问题在计算化学中尤为突出，机器学习势函数（MLFF）的严苛效率要求常迫使忽略长程相互作用。为此，我们提出欧氏快速注意力（EFA），一种专为欧氏数据设计的线性扩展注意力机制，可无缝集成至现有模型架构。核心是新型欧氏旋转位置编码（ERoPE），在保持物理对称性的同时高效编码空间信息。实验表明，EFA能有效捕获多样化的长程效应，使配备EFA的MLFF可准确描述传统方法失效的复杂化学相互作用。
原文摘要 · Abstract (English)
Long-range correlations are essential across numerous machine learning tasks, especially for data embedded in Euclidean space, where the relative positions and orientations of distant components are often critical for accurate predictions. Self-attention offers a compelling mechanism for capturing these global effects, but its quadratic complexity presents a significant practical limitation. This problem is particularly pronounced in computational chemistry, where the stringent efficiency requirements of machine learning force fields (MLFFs) often preclude accurately modeling long-range interactions. To address this, we introduce Euclidean fast attention (EFA), a linear-scaling attention-like mechanism designed for Euclidean data, which can be easily incorporated into existing model architectures. A core component of EFA are novel Euclidean rotary positional encodings (ERoPE), which enable efficient encoding of spatial information while respecting essential physical symmetries. We empirically demonstrate that EFA effectively captures diverse long-range effects, enabling EFA-equipped MLFFs to describe challenging chemical interactions for which conventional MLFFs yield incorrect results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。