提出2比特大模型量化新解码方案，显著提升推理速度与显存效率。
Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights
- 设计多壳层解码融合内核，实现301类码本的高效显存布局。
- 在4.80比特/权重下，解码速度比传统方法快2.15倍，显存占用更低。
- 适合大规模低比特模型部署，尤其对显存敏感的边缘设备有实用价值。
Leech格量化在自有评估协议下实现了最强的2比特模型质量。其核心解码仅处理单一层壳；此前未见支持多层壳解码的实现。本文首次提供完整301类码本的解码路径：离线展开为GPU显存布局，并设计融合去量化与矩阵向量乘的核函数，避免线程发散，经双精度验证无误。显存内码率是独立于磁盘码率的设计维度。在单一进程内测试四种位精确布局，发现二进制位平面在保持带宽不变时，优于一热掩码，达到4.80比特/权重，提速2.15倍（相比FP16）。低于4.3比特/权重时出现不规则数据流；3.6比特以下解码不再依赖移位与掩码操作。四比特（AWQ）与两比特（QTIP）GEMV内核在同一进程中运行。树状结构内核读取字节数仅为所提布局的2.40倍，且提速2.27倍，接近其字节开销上限：时间差距正比于数据流量差距，揭示了超大码本无法使用查表导致的代价。有效性验证表明：树状结构内核超越无权重对照组，因此确定了启动几何的性能下限；在第二级内存层级中，所有格臂均低于FP16性能。固定输出头条件下，该内核-格式路径在4B、8B和14B模型上分别获得1.11倍、1.29倍和1.41倍端到端加速；使用int8输出头时，4B模型达87.0 tok/s，显存仅2.60 GB。质量损失为1.38倍困惑度与14.7点MMLU，在三个尺寸上均持续缩小。
原文摘要 · Abstract (English)
Leech-lattice vector quantization holds the strongest reported 2-bit quality under its own evaluation protocol. Its kernel decodes one shell; we found no implementation of the multi-shell decoder the rate requires. This paper supplies one and measures its serving cost for decode-phase GEMV at batch 1. First, a serving path for the full 301-class codebook: an offline expansion into GPU layouts and a fused dequantize-plus-matvec kernel reading them without warp divergence, verified against f64. Second, the in-VRAM rate is a design axis distinct from the on-disk rate. Four bit-exact layouts timed in one process show binary bit planes beating one-hot masks on size and speed at constant bandwidth (4.80 bits per weight, 2.15x FP16). Below 4.3 bits a second, irregular stream enters; at 3.6 the decode stops being shifts and masks. Third, deployed four-bit (AWQ) and two-bit (QTIP) GEMV kernels run in the same process. The trellis kernel reads 2.40x fewer bytes than our served layout and runs 2.27x faster at near-equal fractions of their byte bounds: the time gap tracks the traffic gap, the price of unfolding a codebook too large for a lookup table. Fourth, the validity envelope: the trellis kernel outruns our no-weights control, so our launch geometry sets that floor, and on a second memory hierarchy every lattice arm falls below FP16. With the output head held identical across arms, the kernel-and-format path gains 1.11x, 1.29x and 1.41x end to end at 4B, 8B and 14B; with an int8 output head the served 4B reaches 87.0 tok/s in 2.60 GB. The quality cost, 1.38x perplexity and 14.7 MMLU points at 4B, shrinks across the three sizes measured.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。