无需复杂操作，轻松实现大模型高效量化，精度损失极小。
SiLQ: Simple Large Language Model Quantization-Aware Training
- 仅通过简单量化训练，不增加额外计算操作。
- 训练成本仅增0.1%却大幅超越现有量化方法。
- 适配多种架构与模块，适合部署在专用加速器上。
大语言模型可通过量化降低推理延迟、模型体积和能耗，从而以更低成本提升用户体验。然而，如何在合理时间内实现精度损失最小的量化仍具挑战，尤其需避免与专用推理加速器不兼容的机制。本文提出一种简单、端到端的量化感知训练方法，总训练预算增加不足0.1%，在多个现代基准上显著优于现有领先量化方法，涵盖基础与指令微调模型。该方法可无缝泛化至不同模型架构，适用于激活值、缓存和权重的量化，且除量化操作外无需引入任何额外计算单元。
原文摘要 · Abstract (English)
Large language models can be quantized to reduce inference time latency, model size, and energy consumption, thereby delivering a better user experience at lower cost. A challenge exists to deliver quantized models with minimal loss of accuracy in reasonable time, and in particular to do so without requiring mechanisms incompatible with specialized inference accelerators. Here, we demonstrate a simple, end-to-end quantization-aware training approach that, with an increase in total model training budget of less than 0.1%, outperforms the leading published quantization methods by large margins on several modern benchmarks, with both base and instruct model variants. The approach easily generalizes across different model architectures, can be applied to activations, cache, and weights, and requires the introduction of no additional operations to the model other than the quantization itself.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。