提出BTC-LLM，实现高效亚1比特大模型压缩，性能损失小且兼容普通硬件。
BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook
- 用可学习变换和二值码本聚类权重，避免稀疏掩码
- 在LLaMA-2-13B上压缩至0.8比特，精度仅降3.1%
- 适合追求极致压缩与推理加速的部署场景
二值量化是极致压缩手段，将权重降至±1以最大化内存与计算效率。现有基于稀疏性的二值化方法虽实现亚1比特压缩，但面临性能下降、掩码管理开销大及硬件兼容性差等问题。本文提出BTC-LLM，一种新型亚1比特大模型量化框架，通过二值模式聚类与可学习权重变换克服上述局限。核心创新包括：（1）自定义距离度量与符号更新机制的二值码本，将重复向量聚类为紧凑索引；（2）可学习变换减少异常值，促进二值权重共享符号模式。该方法无需稀疏掩码，可在标准硬件上高效推理。在LLaMA、Qwen和FBI-LLM系列模型上的实验证明，BTC-LLM在极端压缩（1.11–0.7比特）下达到当前最优效果。特别地，对LLaMA-2-13B压缩至0.8比特时，零样本基准仅损失3.1%精度，同时较FP16快1.6倍。
原文摘要 · Abstract (English)
Binary quantization represents the most extreme form of compression, reducing weights to +/-1 for maximal memory and computational efficiency. While recent sparsity-aware binarization achieves sub-1-bit compression via weight pruning, it faces critical challenges: performance degradation, mask-management overhead, and limited hardware compatibility. In this paper, we present BTC-LLM, a novel sub-1-bit LLM quantization framework that leverages binary pattern clustering and weight transformation to overcome these limitations. Our approach incorporates two key innovations: (1) a Binary Codebook that clusters recurring vectors into compact indices using custom distance metrics and sign-based updates; (2) a Learnable Transformation that reduces outliers and promotes shared sign patterns among binary weights. This eliminates sparse masks, enabling efficient inference on standard hardware. Extensive evaluations across LLaMA, Qwen, and FBI-LLM families demonstrate that BTC-LLM achieves state-of-the-art results in extreme compression (1.11-0.7 bits). Notably, BTC-LLM compressed to 0.8 bits on LLaMA-2-13B maintains high performance, with only a 3.1 percent accuracy drop in zero-shot benchmarks, while delivering a 1.6x speedup over FP16.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。