提出简单高效模型压缩与微调方法，支持1比特量化下仍保持高性能。
ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning
- 通过权重聚类构建码本，分块微调实现高效压缩
- 2-4比特量化性能优于现有方法，1比特也大幅领先
- 可在单张48GB显卡上压缩微调700亿参数大模型
随着大语言模型规模扩大，模型压缩对边缘部署和可访问性至关重要。仅量化权重虽能减小模型体积，但在低比特位宽下易导致性能下降。标准微调不兼容量化模型，现有替代方法通常难以达到全精度微调效果。本文提出ClusComp，一种简单有效的压缩范式：将权重矩阵聚类为码本，并分块微调。ClusComp在2-4比特量化中表现更优，在1比特压缩下仍显著超越现有超低比特方法，且仅需极少微调即可实现高效微调，甚至超过基于量化的现有方法，媲美全精度FP16微调。特别地，该方法可在单张A6000-48GB GPU上完成70B大模型的压缩与微调。
原文摘要 · Abstract (English)
As large language models (LLMs) scale, model compression is crucial for edge deployment and accessibility. Weight-only quantization reduces model size but suffers from performance degradation at lower bit widths. Moreover, standard finetuning is incompatible with quantized models, and alternative methods often fall short of full finetuning. In this paper, we propose ClusComp, a simple yet effective compression paradigm that clusters weight matrices into codebooks and finetunes them block-by-block. ClusComp (1) achieves superior performance in 2-4 bit quantization, (2) pushes compression to 1-bit while outperforming ultra-low-bit methods with minimal finetuning, and (3) enables efficient finetuning, even surpassing existing quantization-based approaches and rivaling full FP16 finetuning. Notably, ClusComp supports compression and finetuning of 70B LLMs on a single A6000-48GB GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。