针对大模型权重设计的分层量化方法，4.5-6比特下接近无损精度。
A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models
- 按层搜索固定与动态码本组合，结合块级选择位和加权余弦选优
- 6.5比特/权重时重建误差低于传统FP8，存储成本降低1.5比特
- 适合部署在带分层查表硬件的场景，尤其关注能效与存储优化
SOP是一种面向大语言模型权重的后训练量化方法，专为支持分层查找表解码的硬件设计，在4.5至6比特每权重范围内实现近乎无损的保真度。该方法融合分层搜索、固定与动态码本对的选择（由块级选择位决定）、块级符号缩放、激活加权余弦选择，以及敏感层的多选背包优化，并引入异常值与稀疏残差修正机制。固定码本包括NF4、BOF4、Split87和SH4；分层优化码本（DD4）存于LUT SRAM中。提出新型硬件高效查找表输出格式（HIF），提升性能、能效与成本表现。在六个开源模型族上验证，推荐的FP6运行点（E2M3sUE4M4，6.5 bpw）相比传统分层POT FP8基线（E4M3，8.0 bpw）在重建误差更低的情况下，存储成本减少1.5 bpw，表明采用块级缩放的小原子结构搭配精细缩放精度可替代传统部署的FP8。完整评估涵盖4.5至6 bpw范围，包含层提升与稀疏残差修正，详见配套论文。
原文摘要 · Abstract (English)
Scaled Outer Product (SOP) is a post-training quantization methodology for large language model weights, designed to deliver near-lossless fidelity at 4.5--6 bits per weight on hardware with per-layer LUT decode. The methodology combines per-layer search of fixed and dynamic codebook pairs selected by a per-block selection bit, signed per-block scales, activation-weighted cosine selection, and multiple-choice knapsack promotion of sensitive layers with outlier and sparse-residual correction. Fixed codebooks include NF4, BOF4, Split87, and SH4; per-layer optimized codebooks (DD4) are hosted in LUT SRAM. A new hardware-efficient LUT output format (HIF) is proposed to improve performance, energy, and cost. Across six open model families, the recommended FP6 operating point (E2M3sUE4M4, 6.5 bpw) achieves lower weight reconstruction error than the conventional per-layer-POT FP8 baseline (E4M3, 8.0 bpw) at 1.5 bpw lower storage cost, demonstrating that block-scaled small atoms with carefully chosen scale precision can replace conventionally-deployed FP8. Full evaluation across the 4.5--6 bpw range, including layer promotion and sparse residual correction, is reported in a companion paper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。