Transformer量化中激活值异常导致精度暴跌，需按通道分配精度应对。
Activation Outliers in Transformer Quantization: Reproduction, Statistical Analysis, and Deployment Tradeoffs
- 发现深层网络激活值呈重尾分布，顶层峰度达271，99%能量集中于1%通道。
- 全模型8位量化使准确率从89.66%降至54.33%，混合精度可恢复至89.42%。
- 按嵌入组分组量化效果显著，但阈值剪裁无效，说明异常值含结构化信息。
对微调后的BERT-base在QNLI上的后训练量化（PTQ）进行可复现的实证研究，发现全局W8A8量化使验证准确率从FP32的89.66%骤降至54.33%，降幅达35.33点。统计分析显示FP32激活值具有强重尾特性，深度增加时加剧：最终层峰度达271，约55%激活能量集中于前1%通道。评估多种缓解策略：混合精度PTQ将准确率恢复至89.42%；按嵌入组（PEG）量化随分组数增加显著提升，三组时为66.12%，四组时达86.18%；而基于百分位的校准（99.0%~99.99%）均无法恢复性能（约50.54%），表明大激活通道承载结构化信号而非噪声。在RTX 3050上部署分析显示各方法延迟（中位数58-59毫秒）与显存占用（约484-486MB）差异极小，凸显硬件感知评估的重要性。结果表明，变压器量化失败主要由残差连接放大的通道主导效应驱动，有效缓解需依赖通道感知的精度分配，而非单纯标量截断。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) of transformers is known to suffer from severe accuracy degradation due to structured activation outliers, as originally analyzed by Bondarenko et al. (EMNLP 2021) in work associated with Qualcomm AI Research. This paper provides a reproducible empirical reproduction and systems-level extension of that phenomenon in BERT-base fine-tuned on QNLI. When global W8A8 quantization is applied, validation accuracy drops sharply from 89.66% (FP32) to 54.33%, a decrease of 35.33 points. Statistical analysis of FP32 activations shows strongly heavy-tailed behavior that intensifies with model depth: kurtosis reaches 271 in the final layers and approximately 55% of activation energy is concentrated in the top 1% of channels. We evaluate several mitigation strategies. Mixed precision PTQ restores accuracy close to the FP32 baseline (89.42%). Per-embedding-group (PEG) quantization shows strong sensitivity to grouping structure, improving accuracy from 66.12% with three groups to 86.18% with four groups. In contrast, percentile-based calibration, even at thresholds between 99.0 and 99.99, fails to recover accuracy (about 50.54%), indicating that large activation channels encode structured signal rather than rare noise. Deployment profiling on an RTX 3050 GPU shows minimal differences in latency and memory usage across methods (median latency about 58-59 ms; VRAM usage about 484-486 MB), highlighting the importance of hardware-aware evaluation. Overall, the results show that PTQ failure in transformers is primarily driven by structured channel dominance amplified through residual connections. Effective mitigation therefore requires channel-aware precision allocation rather than scalar clipping alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。