用矩阵乘法加速低精度线性注意力的逆矩阵计算，提升长序列推理效率。
When Good Enough Is Optimal: Multiplication-Only Matrix Inversion Approximation for Quantized Gated DeltaNet
- 基于截断诺伊曼级数与结构掩码，将逆矩阵计算转为并行矩阵乘法。
- 在低比特整型下实现5倍内核级加速，解码层开销降低20%。
- 适合部署于NPU的低精度长序列模型，兼顾速度与精度。
块状并行线性注意力中的矩阵求逆是长上下文建模的主要瓶颈，尤其在NPU上，基于前代消元的方法并行度低且硬件利用率差。本文提出一种专用于块状线性注意力中严格下三角矩阵的快速矩阵乘法（MatMul）算法。受诺伊曼级数项快速增长及逆矩阵对角集中特性的启发，采用截断诺伊曼展开结合结构掩码与并行残差修正，消除串行依赖。进一步针对低比特整型（INT），缓解重复矩阵幂运算带来的动态范围膨胀问题，并根据块大小自适应调整近似阶数与残差步数，在最小化计算成本的同时保持模型精度。在Qwen3.5系列模型上的实验表明，该方法实现最高5×的内核级加速，解码层开销减少20%，在浮点与低精度推理下均保持准确。本方法为可扩展线性注意力提供了高效且硬件友好的解决方案。
原文摘要 · Abstract (English)
Matrix inversion in chunk-wise parallel linear attention is a major bottleneck for long-context modeling, particularly on NPUs, where forward-substitution-based methods exhibit limited parallelism and poor hardware utilization. We propose a fast, Matrix Multiplication (MatMul)-based algorithm tailored for strictly lower-triangular matrices arising in chunk-wise linear attention. Motivated by the rapid growth of Neumann-series terms and the diagonal concentration of the inverse matrix, we employ a truncated Neumann expansion with structural masking and parallel residual correction to eliminate sequential dependencies. We further extend our method to low-bits INT by mitigating the dynamic range expansion arising from repeated matrix power operations, and adapt the approximation order and residual step to the chunk size to minimize computational cost while preserving the model's accuracy. Experiments on Qwen3.5-family models demonstrate up to 5$\times$ kernel-level speedup and a 20% reduction in decode-layer overhead, while preserving accuracy under both floating-point and low-precision inference. Our method offers an efficient and hardware-friendly solution for scalable linear attention.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。