降低精度反而耗能更多，多步推理中存在量化陷阱。
The Quantization Trap: Breaking Linear Scaling Laws in Multi-Hop Reasoning
- 发现多跳推理中精度降低会导致能耗上升
- 4位量化使720亿模型能耗增加37%
- 适合关注推理效率与硬件部署的工程师
神经网络缩放定律预测：降低数值精度可线性提升计算效率和能效（$E /propto ext{bits}$）。本文揭示，在多跳推理任务中，该定律失效。我们发现‘量化陷阱’——从16位降至8/4位时，净能耗反而上升，推理准确率下降。通过理论分解，归因于硬件截断开销及反量化核的隐性延迟，其在序列推理链中成为主导瓶颈，且能量摊销失败。我们提出临界模型规模 $N^*$，可预测不同模型大小、批处理量和硬件配置下陷阱的深浅，验证范围覆盖0.6B至72B模型，横跨六种GPU架构。结果表明，复杂推理任务中‘越小越好’的行业直觉在数学上是反效率的。
原文摘要 · Abstract (English)
Neural scaling laws provide a predictable recipe for AI advancement: reducing numerical precision should linearly improve computational efficiency and energy profile ($E \propto \mathrm{bits}$). In this paper, we demonstrate that this scaling law breaks in the context of multi-hop reasoning. We reveal a 'quantization trap' where reducing precision from 16-bit to 8/4-bit paradoxically increases net energy consumption while degrading reasoning accuracy. We provide a rigorous theoretical decomposition that attributes this failure to hardware casting overhead, the hidden latency cost of dequantization kernels, which becomes a dominant bottleneck in sequential reasoning chains, as well as to a sequential energy amortization failure. As a result, scaling law breaking is unavoidable in practice. We formalize a Critical Model Scale $N^*$ that predicts when the trap dissolves or deepens as a function of model size, batch size, and hardware configuration, validated across a 120$\times$ range (0.6B--72B) on six GPU architectures. Our findings suggest that the industry's "smaller-is-better" heuristic is mathematically counterproductive for complex reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。