让大模型激活值更易量化，降低推理成本。
Turning LLM Activations Quantization-Friendly
- 分析层间量化误差来源，发现激活值异常点是关键瓶颈。
- 提出通道级缩放结合旋转的混合方法，显著降低量化误差。
- 新量化难度度量工具可直观定位难量化通道，适合模型优化者。
量化通过压缩参数和使用整数运算加速数据移动，有效降低大语言模型（LLMs）的部署成本。然而，要实现整数运算需同时量化权重与激活值，而大模型中显著的异常值会增加量化误差。本文聚焦这些异常值对逐层量化误差的影响，研究平滑与旋转操作如何改变数值分布。主要贡献包括：提出基于通道幅值的新量化难度度量与可视化方法，以及一种先进行通道级缩放再执行旋转的混合策略，并给出其理论优势的数学推导。
原文摘要 · Abstract (English)
Quantization effectively reduces the serving costs of Large Language Models (LLMs) by speeding up data movement through compressed parameters and enabling faster operations via integer arithmetic. However, activating integer arithmetic requires quantizing both weights and activations, which poses challenges due to the significant outliers in LLMs that increase quantization error. In this work, we investigate these outliers with an emphasis on their effect on layer-wise quantization error, then examine how smoothing and rotation transform the observed values. Our primary contributions include introducing a new metric to measure and visualize quantization difficulty based on channel magnitudes, as well as proposing a hybrid approach that applies channel-wise scaling before rotation, supported by a mathematical formulation of its benefits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。