用数学分解法无损加速注意力计算，性能提升显著且无需重训练。
Accelerating Attention with Basis Decomposition
- 基于基分解的矩阵恒等式重构多头投影，实现精确加速。
- 在16B模型上提速34%，权重减少25%，性能下降不足0.02%。
- 适用于各类硬件，可与现有优化协同使用，适合追求高效推理者。
注意力是大语言模型的核心操作。本文提出无损算法重构方法BD Attention（BDA），其基于基分解（BD）的简单矩阵恒等式，将多头投影重组为紧凑形式，保持输出完全一致。与依赖输入输出特性的系统优化（如FlashAttention）不同，BDA提供数学保证的加速，且与架构无关。在DeepSeek-V2-Lite（16B，FP16）上，仅需4秒离线准备，无需重训练，在现代GPU上实现键值投影速度提升34%，权重减少25%，困惑度（PPL）仅上升0.02%（FP16）或0.0004%（FP32），对模型性能影响可忽略。BDA是一种理论精确的无损注意力加速方法，可与现有工程优化互补。代码已开源：https://github.com/abcbdf/basis-decomposition-official。
原文摘要 · Abstract (English)
Attention is a core operation in large language models (LLMs). We present BD Attention (BDA), a lossless algorithmic reformulation of attention. BDA is enabled by a simple matrix identity from Basis Decomposition (BD), which restructures multi-head projections into a compact form while preserving exact outputs. Unlike I/O-aware system optimizations such as FlashAttention, BDA provides a mathematically guaranteed acceleration that is architecture-agnostic. On DeepSeek-V2-Lite (16B, FP16), BDA requires only 4s of offline preparation with no retraining required and, on modern GPUs, achieves 34% faster key/value projections and 25% smaller weights, while increasing perplexity (PPL) by just 0.02% (FP16) or 0.0004% (FP32), a negligible effect on model performance. These results position BDA as a theoretically exact method for lossless attention acceleration that is complementary to existing engineering-level optimizations. Our code is available at https://github.com/abcbdf/basis-decomposition-official.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。