arXiv:2606.12487cs.LG2026-06

动态调整关键层精度,解决4比特量化中激活值崩溃问题。

DynamicPTQ: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics

论文配图:DynamicPTQ: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics
图 1 · 摘自论文原文
  • 根据残差流动态识别敏感层,仅对这些层用8比特量化。
  • 在LLaMA-2/3上实现1.05~1.07倍吞吐提升,困惑度更低。
  • 适合追求高效低比特推理的模型部署开发者。

后训练量化(PTQ)对大语言模型高效推理至关重要,但在权重、激活值和键值缓存均量化为4比特时,可靠量化激活值仍具挑战性。核心难点在于大规模激活值的极端值主导激活范围并放大量化误差。现有方法主要通过正交旋转、仿射缩放等变换平滑处理,但忽略了残差流的跨层动态特性。本文发现大规模激活值在网络深度呈现阶段性出现与消失模式,引发显著残差变化,导致新注入的层内更新主导4比特量化尺度,削弱历史残差信息。为此,我们引入跳跃比(Jump Ratio)和历史特征信噪比(Historical Feature SNR)进行表征。结果表明,静态变换平滑无法完全缓解由跨层残差变化引起的动态量化不稳定性。基于此,提出动态后训练量化策略DynamicPTQ,实现相位感知的混合精度激活量化。该方法识别残差流动态中的敏感层,仅对其分配8比特激活精度,其余保持4比特。可直接集成至QuaRot、SpinQuant、FlatQuant等强基准方法。在LLaMA-2与LLaMA-3上的实验表明,DynamicPTQ在W4A4KV4量化下持续提升困惑度与零样本问答性能,同时带来1.05至1.07倍吞吐提升,内存开销适中。这为鲁棒低比特大模型推理提供了可行路径。

原文摘要 · Abstract (English)

Post-training quantization (PTQ) is essential for efficient large language model inference, but reliably quantizing activations remains challenging when weights, activations, and KV caches are all quantized to 4-bit precision. A key difficulty lies in massive activations, whose extreme values dominate the activation range and amplify quantization errors. State-of-the-art methods mainly mitigate massive activations through transformation-based smoothing, such as orthogonal rotations and affine scaling, but overlook the cross-layer dynamics of the residual stream. In this paper, we show that massive activations emerge and disappear in a phase-wise pattern across network depth, triggering large residual changes. These changes cause newly injected layer-wise updates to dominate the 4-bit quantization scale and weaken historical residual information. To characterize this behavior, we introduce Jump Ratio and Historical Feature SNR. This suggests that static transformation-based smoothing cannot fully resolve dynamic quantization instability caused by cross-layer residual changes. Based on this analysis, we propose DynamicPTQ, a Dynamic Post-Training Quantization policy for phase-aware mixed-precision activation quantization. DynamicPTQ identifies quantization-sensitive layers from residual-stream dynamics and assigns 8-bit activation precision only to these layers, while keeping weights, KV caches, and other activations in 4-bit precision. It can be directly integrated with strong PTQ baselines such as QuaRot, SpinQuant, and FlatQuant. Experiments on LLaMA-2 and LLaMA-3 show that DynamicPTQ consistently improves perplexity and zero-shot QA performance under W4A4KV4 quantization, while achieving 1.05 to 1.07 times throughput improvement with modest memory overhead. These results demonstrate a practical path toward robust low-bit LLM inference.

量化大模型动态精度推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。