4-bit量化看似无损,实则放大了大模型代理的已有错误。
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
- 通过缩小错误预算,发现4比特量化使错误量增加2.5倍
- 错误类型未新增,但电信领域工具名幻觉上升17.6点/任务
- 只需监测每通道错误率和缩减预算即可暴露隐藏损伤
后训练4比特量化被广泛认为几乎无损。我们在多轮、调用工具的智能体上检验此说法,发现其在关键场景下并不成立。在τ²-bench上,对两种开源模型族(密集与MoE变体)及两个领域(8个任务单元,每个456个实验),在16、8、4比特权重下,标准指标显示量化无影响:无单元得分变化通过多重比较校正,最大过程损伤的置信区间在±7.5分内。然而过程分析揭示不同事实:量化将全精度下已存在的错误(如电信领域的工具名幻觉)放大至2.5倍,错误量上升17.6分/任务,且未引入新错误。错误集合在各精度下一致(秩相关≥0.94,仅0.18%新事件)。分数持平是因为基准测试的十错误预算吸收了额外错误。将预算压缩至两个错误后,17分差距重现,且仅出现在量化增加错误量的单元中,符合掩蔽假说。针对电信模型的定向修复提示在所有精度下精确消除损伤,仅作用于受损区域。诊断方法——每通道错误率与缩减预算下的成功率——均来自现有日志数据,建议与任务奖励一同报告。
原文摘要 · Abstract (English)
Post-training quantization to 4-bit weights is widely reported to be nearly lossless. We test this claim for multi-turn, tool-calling agents, where it now matters most. On $τ^2$-bench, across two open-weight model families in dense and MoE variants and two domains (eight cells, 456 episodes each, at 16-, 8-, and 4-bit weights), quantization indeed looks free on the standard metric. No cell shows a score change that survives multiple-comparison correction, and in the cell that carries the largest process damage, equivalence testing bounds the change within $\pm$7.5 points. The process tells a different story. Quantization amplifies the failure the model already exhibits at full precision (tool-name hallucination in telecom, with the same directional trend in retail entity errors) by up to 2.5$\times$ in volume (+17.6 points per task), while creating essentially no new failures. The failure set is the same at every precision (rank correlation $\geq$ 0.94, 0.18% novel events). The score stays flat because the benchmark's ten-error budget absorbs the extra failures. Shrinking the budget to two errors re-exposes a score gap of 17 points, and it does so only in the one cell where quantization added error volume, exactly as the masking account predicts. A targeted error-repair prompt, run for five telecom models at every precision, removes the damage exactly and only where it lives. Both diagnostics, the per-channel error rate and success under a shrinking budget, come from logs benchmarks already collect; we suggest reporting them alongside task reward.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。