用视频编码技术压缩大模型，效率更高且无需额外校准数据。
LLMCodec: Adapting Video Codecs for Efficient Weight Compression of Large Language Models
- 将视频编码器VVC/H.266与量化结合，直接压缩模型权重矩阵。
- 在LLaMA-3-8B上2比特精度下，困惑度降低1.5倍，任务准确率提升21%。
- 不依赖微调或校准数据，适配多种模型和张量类型，通用性强。
大语言模型（LLMs）的快速发展带来了自然语言处理的巨大进步，但其日益增长的规模也带来了存储、传输和部署方面的严峻挑战。尽管已有大量工作致力于模型压缩和量化，但现有方法通常依赖微调或校准数据，对不同张量类型的泛化能力有限。本文认为，视频编码器因其对矩阵结构数据的天然兼容性、可配置的压缩策略以及高度优化的现成实现，是解决LLM压缩问题的有力方案。为此，我们提出LLMCodec，一种基于视频编码器的LLM压缩方法，融合仿射量化与最新的VVC/H.266视频编码标准。除VVC外，我们还对比了多种视频编码器及编码配置，评估其对压缩性能的影响。实验表明，LLMCodec具有良好的鲁棒性和通用性。值得注意的是，在LLaMA-3-8B模型中，2比特精度下，相比现有方法，其困惑度降低超过1.5倍，下游任务准确率提升21%。
原文摘要 · Abstract (English)
The rapid development of large language models(LLMs) has led to remarkable advances in natural language processing. However, the increasing scale of these models introduces substantial challenges in terms of storage, transmission, and deployment. Though great efforts have been devoted to model compression and quantization, existing methods often rely on fine-tuning or calibration data, which exhibit limited generalization across different tensor types. In this paper, we argue that video codecs offer a promising solution for LLM compression, due to their inherent compatibility with matrix structured data, configurable compression strategies, and the availability of highly optimized, off-the-shelf implementations. Therefore, we present LLMCodec, a video codec-based LLM compression method that integrates affine quantization with the recent VVC/H.266 video codec. Beyond VVC, we further compare a range of video codecs and encoding profiles to evaluate their impact on compression performance. Experiments on different models demonstrate the robustness and generality of LLMCodec. Notably, on LLaMA-3-8B at 2-bit precision, LLMCodec reduces perplexity by over 1.5x and improves downstream task accuracy by 21% compared with the existing method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。