通过分离敏感权重方向，提升低比特大模型推理精度。
Technical Report: Activation Residual Hessian Quantization (ARHQ) for Low-Bit LLM Quantization
- 将激活量化误差构建为输入侧残差海森矩阵，识别敏感权重方向。
- 在Qwen3-4B模型上，大幅改善层内信噪比，推理性能损失小。
- 适合需要高精度低比特部署的大模型研究者使用。
我们提出激活残差海森量化（ARHQ），一种后训练权重拆分方法，用于缓解低比特激活-权重量化中的误差传播问题。通过构建基于激活量化残差（G_x）的输入侧残差海森矩阵，ARHQ 通过闭式截断SVD对缩放后的权重矩阵 W G^{1/2}_x 进行分析，精确识别并分离出误差敏感的权重方向至高精度低秩分支。在 Qwen3-4B-Thinking-2507 模型上的实验表明，ARHQ 显著提升了层内信噪比（SNR），即使在激进量化条件下，也能保持下游推理任务（ZebraLogic）的性能。代码已开源：https://github.com/BeautMoonQ/ARHQ。
原文摘要 · Abstract (English)
We present Activation Residual Hessian Quantization (ARHQ), a post-training weight splitting method designed to mitigate error propagation in low-bit activation-weight quantization. By constructing an input-side residual Hessian from activation quantization residuals (G_x), ARHQ analytically identifies and isolates error-sensitive weight directions into a high-precision low-rank branch. This is achieved via a closed-form truncated SVD on the scaled weight matrix W G^{1/2}_x . Experimental results on Qwen3-4B-Thinking-2507 demonstrate that ARHQ significantly improves layer-wise SNR and preserves downstream reasoning performance on ZebraLogic even under aggressive quantization. The code is available at https://github.com/BeautMoonQ/ARHQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。