提出CRVQ方法，实现1比特大模型压缩接近无损效果。
CRVQ: Channel-Relaxed Vector Quantization for Extreme Compression of LLMs
- 仅选取关键权重通道并重新排序，降低量化复杂度。
- 使用扩展码本放松通道约束，提升1比特压缩性能38.9%。
- 支持灵活位宽配置，适配不同硬件部署需求。
强大的大语言模型（LLMs）正面临更低计算成本的部署需求，以在资源受限设备上运行。后训练量化（PTQ）成为实现这一目标的关键技术，最佳方法已可将权重压缩至平均低于2比特。本文提出通道松弛向量量化（CRVQ），一种显著提升现有PTQ基线性能的新方法，额外消耗极少量比特。该方法通过两大创新实现：（1）精心选择并重排少量关键权重通道；（2）利用扩展码本松弛关键通道的约束。实验表明，相比当前最强的亚2比特PTQ基线，本方法性能提升达38.9%，推动1比特压缩更接近无损。此外，该方法支持量化位宽与性能的灵活定制，为多样硬件平台提供更广部署选项。
原文摘要 · Abstract (English)
Powerful large language models (LLMs) are increasingly expected to be deployed with lower computational costs, enabling their capabilities on resource-constrained devices. Post-training quantization (PTQ) has emerged as a star approach to achieve this ambition, with best methods compressing weights to less than 2 bit on average. In this paper, we propose Channel-Relaxed Vector Quantization (CRVQ), a novel technique that significantly improves the performance of PTQ baselines at the cost of only minimal additional bits. This state-of-the-art extreme compression method achieves its results through two key innovations: (1) carefully selecting and reordering a very small subset of critical weight channels, and (2) leveraging extended codebooks to relax the constraint of critical channels. With our method, we demonstrate a 38.9\% improvement over the current strongest sub-2-bit PTQ baseline, enabling nearer lossless 1-bit compression. Furthermore, our approach offers flexible customization of quantization bit-width and performance, providing a wider range of deployment options for diverse hardware platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。