改进大模型量化中的通道缩放策略，降低异常值带来的误差。
Trainable Smooth-Rotation Transforms with Learned Channel Scales for LLM Quantization
- 用高分位数替代最大值统计，增强对异常值的鲁棒性。
- 联合优化缩放参数与分位数，使层内误差下降18.5%。
- 适用于需高效部署的大模型，尤其适合低比特量化场景。
后训练量化（PTQ）是降低大语言模型（LLM）服务成本的重要方法，但激活量化仍因异常值主导的通道导致较大量化误差。本文研究发现，基于缩放的等价变换中过度迁移可能是问题根源。提出一种针对SmoothRot类变换的分位数鲁棒缩放策略，以高分位数替代最大值作为激活统计量，并引入约束梯度优化通道缩放。在LLaMA-3.2-1B模型上采用W4A4量化时，仅分位数搜索使选定层误差降低11.1%，联合（alpha, q）搜索提升至12%，训练可达到18.5%。将最优选定层策略应用于所有解码器块下投影层，全层平均误差由97.51降至78.08（降幅19.9%）。结果表明，稳健的迁移控制与轻量级缩放学习在保持等价变换框架的同时带来持续增益。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) is one of the most practical ways to reduce the serving cost of Large Language Models (LLMs), but activation quantization remains difficult because outlier-dominated channels lead to large quantization errors. This paper investigates whether part of this degradation is caused by over-migration in scaling-based equivalent transformations. We introduce a quantile-robust scaling policy for SmoothRot-style transforms by replacing max-based activation statistics with high quantiles, and we complement it with constrained gradient-based optimization of channel scales. On LLaMA-3.2-1B under W4A4 quantization, quantile-only policy search improves selected-layer error by 11.1% over the SmoothRot baseline, joint (alpha, q) search improves it by 12%, and training reaches 18.5%. Replaying the best selected-layer policy on all decoder-block down-projection layers reduces the corresponding full-layer mean error from 97.51 to 78.08 (19.9%). The results show that robust migration control and lightweight scale learning provide consistent gains over max-based fixed policies while preserving the equivalent-transform framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。