让大模型量化比特数连续可调，精准适配内存限制
LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection
- 通过升维投影实现比特数连续调节，突破整数比特限制
- 70B模型压缩至2.4比特仍性能优于现有2比特模型
- 仅用1比特量化器，兼顾高效与硬件友好性
现有量化方法受限于固定整数比特（如2、3比特），导致大语言模型无法在特定内存预算下实现最优部署。为弥合这一差距，我们提出LiftQuant框架，实现真正的帕累托最优部署下的连续比特位控制。其核心是“升维-投影”机制：通过将低维权重向量投影到高维升维空间中的1比特网格，有效比特数由升维维度与原维度之比决定，使比特数可近似连续调节。该投影生成结构化但非均匀的码本，兼具向量量化（VQ）的表达能力。与VQ相比，LiftQuant解码路径仅依赖线性变换和1比特均匀量化器，保持硬件友好性。此灵活性极具价值：70B模型可被压缩至2.4比特，精确适配24GB显存，在相同设备上性能显著超越当前最先进的2比特模型。
原文摘要 · Abstract (English)
Existing quantization methods are fundamentally limited by rigid, integer-based bit-widths (e.g., 2, 3-bit), resulting in a ``deployment gap" where Large Language Models cannot be optimally fitted to specific memory budgets. To bridge this gap, we introduce LiftQuant, a novel framework that enables continuous bit-width control for true Pareto-optimal deployment. The core innovation is a ``lift-then-project" mechanism which approximates low-dimensional weight vectors by projecting a simple 1-bit lattice from a higher-dimensional ``lifted" space. Crucially, the effective bit-width is determined simply by the ratio of the lifted dimension to the original dimension, which allows the bit-width to be tuned quasi-continuous as the dimension is a flexible structural parameter. This projection generates a structured yet non-uniform codebook, capturing the expressive power of Vector Quantization (VQ). While beneficial over VQ, LiftQuant's decoding path relies solely on linear transformations and 1-bit uniform quantizers, retaining hardware-friendly nature. This flexibility is transformative: LiftQuant enables a 70B LLM to be compressed to 2.4 bits to precisely fit a 24GB GPU, where its performance significantly surpasses state-of-the-art 2-bit models fitted on the same device. Our code and ckpt is available at https://github.com/Heliulu/LiftQuant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。