用未来层激活指导量化，提升大模型压缩精度与稳定性。
Enhancing Post-Training Quantization via Future Activation Awareness
- 利用未来层激活信息优化当前层量化参数
- 在多个数据集上超越已有方法，误差更低且更稳定
- 无需反向传播或调参，适合边缘设备部署
后训练量化（PTQ）是一种无需微调即可压缩大语言模型的常用方法，通常基于当前层激活设定量化超参数（如缩放因子）。尽管高效，该方法易受量化偏差和误差累积影响，尤其在校准数据有偏时表现不佳。为此，我们提出未来感知量化（FAQ），利用未来层激活指导当前层量化，更准确识别并保留关键权重，降低对校准噪声的敏感性。进一步引入窗口式预览机制，软性聚合多层未来激活，避免过度依赖单一层。为减少计算开销，采用预先搜索的配置，几乎不增加额外成本。实验表明，FAQ在多种数据集上持续优于现有方法，且无需反向传播、数据重构或调参，非常适合边缘部署。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) is a widely used method to compress large language models (LLMs) without fine-tuning. It typically sets quantization hyperparameters (e.g., scaling factors) based on current-layer activations. Although this method is efficient, it suffers from quantization bias and error accumulation, resulting in suboptimal and unstable quantization, especially when the calibration data is biased. To overcome these issues, we propose Future-Aware Quantization (FAQ), which leverages future-layer activations to guide quantization. This allows better identification and preservation of important weights, while reducing sensitivity to calibration noise. We further introduce a window-wise preview mechanism to softly aggregate multiple future-layer activations, mitigating over-reliance on any single layer. To avoid expensive greedy search, we use a pre-searched configuration to minimize overhead. Experiments show that FAQ consistently outperforms prior methods with negligible extra cost, requiring no backward passes, data reconstruction, or tuning, making it well-suited for edge deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。