提出球形稀疏注意力，高效处理大规模无序点云几何数据
BSA: Ball Sparse Attention for Large-scale Geometries
- 用球树结构构建点集邻域，实现稀疏注意力的不规则几何适配
- 在气流压力预测任务中达到全注意力精度，计算复杂度低于二次方
- 适合需要处理海量不规则空间数据的研究者与工程师
自注意力机制随输入规模呈平方级增长，难以应用于大规模物理系统。尽管稀疏注意力是可行替代方案，但其多针对文本或图像等规则结构设计，难以适用于不规则几何。本文提出球形稀疏注意力（BSA），通过引入Erwin Transformer中的球树结构，将原生稀疏注意力（NSA）适配到无序点集上。我们修改了NSA组件以支持基于球体的邻域建模，实现了全局感受野且计算成本低于二次方。在气流压力预测任务中，模型精度接近全注意力，同时显著降低理论计算复杂度。代码已开源：https://github.com/britacatalin/bsa。
原文摘要 · Abstract (English)
Self-attention scales quadratically with input size, limiting its use for large-scale physical systems. Although sparse attention mechanisms provide a viable alternative, they are primarily designed for regular structures such as text or images, making them inapplicable for irregular geometries. In this work, we present Ball Sparse Attention (BSA), which adapts Native Sparse Attention (NSA) (Yuan et al., 2025) to unordered point sets by imposing regularity using the Ball Tree structure from the Erwin Transformer (Zhdanov et al., 2025). We modify NSA's components to work with ball-based neighborhoods, yielding a global receptive field at sub-quadratic cost. On an airflow pressure prediction task, we achieve accuracy comparable to Full Attention while significantly reducing the theoretical computational complexity. Our implementation is available at https://github.com/britacatalin/bsa.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。