让Transformer注意力变得稀疏且可解释，性能不降反而更清晰。
Intrinsically Interpretable Attention via Sparse Post-Training
- 后训练时用稀疏正则化约束，让注意力连接减少到原规模的0.4%。
- 稀疏化后任务电路组件和连接数减少超100倍，结构更简洁。
- 简化注意力归属，统一特征与电路视角，适合模型可解释性研究者。
我们提出一种简单的后训练方法，可在不损失性能的前提下使Transformer注意力稀疏化。在不超过70亿参数的模型上，通过灵活的稀疏正则化与受限损失目标结合，可在保持原始预训练损失的同时，将注意力连接减少至约0.4%。不同于为计算效率设计的稀疏注意力方法，本方法将稀疏性作为结构先验：既保留模型能力，又揭示出更有序、可解释的连接模式。我们发现这种局部稀疏性会引发全局电路简化——任务特定电路所用的注意力头与MLP组件显著减少，连接边数最多减少100倍。此外，借助跨层转换器（cross-layer transcoders），我们证明稀疏注意力大幅简化了注意力归因，实现了特征导向与电路导向视角的统一。结果表明，Transformer注意力可实现数量级稀疏化，暗示其大量计算冗余，稀疏性或可成为构建更结构化、可解释模型的指导原则。
原文摘要 · Abstract (English)
We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance. Applying a flexible sparsity regularisation under a constrained-loss objective, we show on models up to 7B parameters that it is possible to retain the original pretraining loss while reducing attention connectivity to $\approx 0.4 \%$ of its edges. Unlike sparse-attention methods designed for computational efficiency, our approach leverages sparsity as a structural prior: it preserves capability while exposing a more organized and interpretable connectivity pattern. We find that this local sparsity cascades into global circuit simplification: task-specific circuits involve far fewer components (attention heads and MLPs) with up to 100x fewer edges connecting them. Additionally, using cross-layer transcoders, we show that sparse attention substantially simplifies attention attribution, enabling a unified view of feature-based and circuit-based perspectives. These results demonstrate that transformer attention can be made orders of magnitude sparser, suggesting that much of its computation is redundant and that sparsity may serve as a guiding principle for more structured and interpretable models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。