用熵编码实现700亿参数模型10分钟内无数据压缩
Float8@2bits: Entropy Coding Enables Data-Free Model Compression
- 通过熵编码分离精度与存储开销，突破4比特以下压缩瓶颈
- 70B模型压缩耗时<10分钟，性能媲美依赖校准数据的方法
- 无需训练数据，适合资源受限场景下的极端压缩需求
后训练压缩目前分为两种截然不同的范式：一方面，快速、无数据、模型无关的方法（如NF4或HQQ）虽易用但低于4比特时会出现功能崩溃；另一方面，依赖校准数据或大量恢复训练的技术虽保真度高，却带来高昂计算成本，且在数据分布变化下鲁棒性不确定。我们提出EntQuant框架，融合两类方法的优势。通过熵编码将数值精度与存储成本解耦，使70B参数模型可在10分钟内完成压缩。实验表明，EntQuant不仅在标准评估集和模型上达到当前最优性能，还在指令微调模型的复杂基准上保持功能性，且推理开销极小。
原文摘要 · Abstract (English)
Post-training compression is currently divided into two contrasting regimes. On the one hand, fast, data-free, and model-agnostic methods (e.g., NF4 or HQQ) offer maximum accessibility but suffer from functional collapse at extreme bit-rates below 4 bits. On the other hand, techniques leveraging calibration data or extensive recovery training achieve superior fidelity but impose high computational constraints and face uncertain robustness under data distribution shifts. We introduce EntQuant, a framework that unites the advantages of these distinct paradigms. By matching the performance of data-dependent methods with the speed and universality of data-free techniques, EntQuant enables practical utility in the extreme compression regime. Our method decouples numerical precision from storage cost via entropy coding, compressing a 70B parameter model in less than 10 minutes. We demonstrate that EntQuant does not only achieve state-of-the-art results on standard evaluation sets and models, but also retains functional performance on more complex benchmarks with instruction-tuned models, all at modest inference overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。