用物理启发的分区方法，让注意力模型更快更省资源地识别粒子喷注
Spatially Aware Linear Transformer (SAL-T) for Particle Jet Tagging
- 基于粒子动量特征分块计算注意力，提升物理相关区域的捕捉效率
- 在喷注分类任务中达到全注意力模型效果，仅需其1/10的计算资源
- 适合高能物理实验中实时处理海量数据，兼顾精度与速度
Transformer 在高能粒子碰撞中能有效捕捉全局与局部相关性，但在如欧洲核子研究中心（CERN LHC）这类高数据吞吐场景中存在部署难题。其二次复杂度导致推理时资源消耗大、延迟高。为此，我们提出空间感知线性 Transformer（SAL-T），在 linformer 架构基础上引入物理启发的空间分区机制，实现线性注意力计算。该方法依据粒子的运动学特征对粒子进行空间划分，仅在具有物理意义的区域间计算注意力；同时结合卷积层捕获局部关联，借鉴喷注物理先验。SAL-T 在喷注分类任务中超越标准 linformer，性能接近全注意力 Transformer，且推理资源消耗大幅降低、延迟更低。在通用点云分类数据集 ModelNet10 上的实验也验证了该趋势。代码已开源：https://github.com/aaronw5/SAL-T4HEP。
原文摘要 · Abstract (English)
Transformers are very effective in capturing both global and local correlations within high-energy particle collisions, but they present deployment challenges in high-data-throughput environments, such as the CERN LHC. The quadratic complexity of transformer models demands substantial resources and increases latency during inference. In order to address these issues, we introduce the Spatially Aware Linear Transformer (SAL-T), a physics-inspired enhancement of the linformer architecture that maintains linear attention. Our method incorporates spatially aware partitioning of particles based on kinematic features, thereby computing attention between regions of physical significance. Additionally, we employ convolutional layers to capture local correlations, informed by insights from jet physics. In addition to outperforming the standard linformer in jet classification tasks, SAL-T also achieves classification results comparable to full-attention transformers, while using considerably fewer resources with lower latency during inference. Experiments on a generic point cloud classification dataset (ModelNet10) further confirm this trend. Our code is available at https://github.com/aaronw5/SAL-T4HEP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。