提出新方法解决1比特量化模型输出失真问题,显著提升性能。
Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models
- 通过输出对齐优化,避免传统权重对齐的误差累积
- 在多个LLM上实现比现有方法更高的准确率,如GPT-2在100个样本上提升4.3%
- 适合需部署轻量大模型的边缘设备场景
大语言模型(LLMs)在众多自然语言任务中表现优异,但其庞大体积限制了在资源受限设备上的部署。为降低计算与内存负担,压缩技术如量化、剪枝和知识蒸馏被广泛研究。其中,后训练量化(PTQ)因无需重训练且仅需少量校准数据,具有高效性优势。近期研究表明,接近4比特的量化可保持大部分原始性能,但1比特量化仍极具挑战。传统1比特量化多采用权重驱动策略,即通过匹配全精度参数确定二值权重,但该目标与量化模型实际输出目标不一致。更优策略应为输出驱动,即最小化校准数据下的输出差异。然而,直接使用输出驱动方法在1比特下效果反而更差。本文揭示其根本原因在于层间误差累积及表示空间的各向异性失真。基于此,我们提出一种新的1比特PTQ方法,显式解决上述问题,同时保持高效率。大量实验表明,该方法持续优于现有1比特PTQ方案。
原文摘要 · Abstract (English)
Large Language Models (LLMs) deliver strong performance across a wide range of NLP tasks, but their massive sizes hinder deployment on resource-constrained devices. To reduce their computational and memory burden, various compression techniques have been proposed, including quantization, pruning, and knowledge distillation. Among these, post-training quantization (PTQ) is widely adopted for its efficiency, as it requires no retraining and only a small dataset for calibration, enabling low-cost deployment. Recent advances for post-training quantization have demonstrated that even near 4-bit methods can maintain most of the original model performance. However, 1-bit quantization remains particularly challenging. A common strategy in 1-bit quantization is to determine binary weights by matching full-precision parameters, following a weight-driven criterion. However, this objective is not directly aligned with the quantized model's objective, which is to preserve the model's output behavior under the impact of quantization. A natural alternative is to adopt output-driven criteria that minimize discrepancies in model outputs using calibration data. Surprisingly, naive output-driven approaches often perform even worse in the 1-bit regime. In this paper, we show that this failure arises from two fundamental issues: error accumulation across layers and, more critically, \emph{anisotropic distortion} of the representation space. Based on these insights, we propose a novel PTQ method for 1-bit LLMs that explicitly addresses these issues while maintaining computational efficiency. Extensive experiments demonstrate that our approach consistently outperforms existing 1-bit PTQ methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。