arXiv:2509.23809cs.LGcs.AI2025-09被引 11

解决大模型二值量化中权重困在死区的问题,提升精度和速度。

Tequila: Trapping-free Ternary Quantization for Large Language Models

  • 将困在死区的权重转为动态偏置,恢复其梯度更新能力。
  • 在ARC基准上比当前最优方法高4%以上,接近全精度性能。
  • 几乎无推理开销,适合边缘设备部署大型语言模型。

量化技术对大语言模型在边缘设备上的部署至关重要。现有方法多依赖缺乏硬件支持的混合精度乘法,难以实现。三值权重量化通过将权重限制在{-1, 0, 1},用高效加法替代昂贵乘法,但过度压缩导致显著精度下降,即使经过大量数据的量化感知训练也难改善。我们发现核心问题是死区陷阱:大量权重被卡在死区边界,因仅接收噪声且无信息的梯度,无法稳定脱离,严重阻碍模型容量与优化。为此,提出Tequila,一种免死区陷阱的量化优化方法,通过将被困权重重用于动态偏置,使其在前向传播中提供连续信号,并在反向传播中获得直接有意义的梯度,从而增强模型能力和优化效率,几乎不增加推理开销。大量评估表明,Tequila在五个基准上均优于当前最优三值量化方法。特别是在ARC基准上,精度较最优基线提升超过4%,与全精度性能差距小于1%,推理速度提升3.0倍。因此,Tequila为资源受限环境下的先进大模型部署提供了高效实用的方案。代码已开源:https://github.com/Tencent/AngelSlim。

原文摘要 · Abstract (English)

Quantization techniques are essential for the deployment of Large Language Models (LLMs) on edge devices. However, prevailing methods often rely on mixed-precision multiplication that lacks efficient hardware support, making it not feasible. Ternary weight quantization addresses this by constraining weights to {-1, 0, 1}, replacing expensive multiplications with hardware-efficient additions. However, such aggressive compression leads to significant accuracy degradation, even after costly quantization-aware training with massive data. We identify the core issue as deadzone trapping: a large number of weights are trapped at the deadzone boundary. This occurs because these weights receive only noisy, uninformative gradients, preventing stable escape from the deadzone and severely impeding model capacity and optimization. To address this issue, we propose Tequila, a trapping-free quantization optimization method that reactivates deadzone-trapped weights by repurposing them as dynamic biases. This allows the repurposed weights to provide a continuous signal in the forward pass and, critically, receive direct, meaningful gradient signals during backpropagation, thereby enhancing model capacity and optimization with nearly zero inference overhead. Extensive evaluations demonstrate that Tequila outperforms state-of-the-art (SOTA) ternary quantization methods across five benchmarks. Specifically, on the ARC benchmark, it achieves >4% accuracy gain over the SOTA baseline, nearly matching full-precision performance (within <1% gap) with a 3.0x inference speedup. Consequently, Tequila offers a highly practical and efficient implementation for the deployment of advanced LLMs in resource-constrained environments. The code is available at https://github.com/Tencent/AngelSlim.

量化大模型边缘计算三值量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。