4bit大模型量化中，变换设计存在根本矛盾，需结合格式协同优化。
Transforms for LLM Quantization: The Great Inversion and Format Co-Design
- 提出'大反转'原理：压缩与分配策略目标相反，需权衡能量集中与组内平坦化。
- 实证200篇论文，发现非均匀格式（如NVFP4）削弱旋转激励，影响量化性能。
- 为不同部署场景提供变换方法选择指南，揭示格式与策略协同设计的开放问题。
当前高效的4比特大模型量化流程普遍采用线性、保函数变换（如旋转、缩放、置换、非正交仿射），使异常值更适应分组尺度，再进行量化。然而，该阶段尚无系统性综述，且文献多在重复早期理论。本文识别并形式化其核心原则——“大反转”：分配灵活编码鼓励能量集中，而部署矩阵指令中的分组共享尺度量化则奖励组内平坦化。经典变换编码（1963年：去相关、分配比特、量化）在高率下以卡尔胡宁-洛埃变换最小化高斯源失真；但实际运算中每组仅用一个绝对最大尺度，比特均分，无分配机制，此时目标鼓励平坦化，接近哈达玛不相干性。我们证明二者在组内主要化意义下对立：各自在特定目标下最优，但通用谱下无法保证最优性转移。第二维度是数值格式：非均匀FP4网格减弱平坦化收益，MXFP4的幂次块尺度仍驱动块内旋转，而NVFP4的尾数携带尺度基本消除此倾向，故最优变换方向依赖于分配机制与格式协同。本文调研至2026年6月的200篇工作，按结构、数据感知度、搜索/构造方式、运行时开销分类43种变换方法；记录其与GPTQ量化组合情况；提炼部署场景下的首选建议，并指出待解决问题。
原文摘要 · Abstract (English)
Most competitive 4-bit LLM research pipelines now open the same way: apply a linear, function-preserving transform (rotation, scaling, permutation, non-orthogonal affine) so the outlier mass sits more favorably against the group scales, and only then round. Yet we are aware of no survey dedicated to this transform stage, and its literature is quietly re-deriving an older theory. We identify and formalize the principle that organizes it, the Great Inversion: allocation-flexible coding rewards energy concentration, whereas the grouped shared-scale quantization a deployed matrix instruction performs rewards within-group flattening. Classical transform coding (1963: decorrelate, allocate bits, quantize) spends different bits per coordinate at a fixed total rate; for a Gaussian source at high rate the Karhunen-Loeve transform's concentration minimizes distortion. A deployed operand tile instead carries one absolute-maximum scale per group and equal bits everywhere, with no allocation; on a uniform grid that objective rewards flattening, approached by Hadamard incoherence. We prove that opposition under within-group majorization: the prescriptions point in opposite directions, each backed by a proof against its own objective, and for a generic spectrum no optimality guarantee transfers. A second axis is the number format: the non-uniform FP4 grid makes flattening buy less, MXFP4's power-of-two block scale still rewards a rotation confined to that block, and NVFP4's mantissa-carrying scale largely removes that pull, so the target pole depends jointly on allocation regime and format. We survey 200 works to a June 2026 cutoff; classify 43 transform methods by structure, data-awareness, searched-versus-constructed, and runtime cost; record, where reported, how they compose with GPTQ rounding; distill a first-choice guide by deployment regime; and close with the open problems it exposes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。