arXiv:2605.18475cs.LGcs.AI2026-05

提出一种无需训练的混合精度量化方法,可任意预算下快速部署大模型。

GAMMA: Global Bit Allocation for Mixed-Precision Models under Arbitrary Budgets

论文配图:GAMMA: Global Bit Allocation for Mixed-Precision Models under Arbitrary Budgets
图 1 · 摘自论文原文
  • 后训练阶段学习模块敏感度,通过整数规划实现精确预算分配。
  • 在8B–32B模型上最高提升12.99分,2.5比特平均精度媲美3比特固定精度。
  • 单次训练可适配任意预算,部署耗时从小时级降至分钟级。

混合精度量化通过为敏感模块分配更多比特,提升了大语言模型(LLM)在预算与精度间的权衡。然而,在千亿参数规模上自动化分配面临独特挑战:可学习方法需量化感知训练,对百亿参数模型不可行;免训练替代方案依赖静态代理指标,忽略模块间交互且需按目标预算重新计算;搜索方法成本高且无法保证精确满足预算。我们提出GAMMA,一种量化器无关的框架,完全在后训练流程中学习模块级精度偏好。GAMMA在增强拉格朗日约束下优化教师强制的隐状态重建目标,并通过整数规划将学习到的偏好投影为精确预算可行的离散分配。关键特性为分数复用:因学习偏好编码的是稳定敏感度排序而非特定预算权重,单次训练即可支持任意部署目标,仅需重解整数规划,使每预算适配时间从数小时降至几分钟。在Llama和Qwen模型(8B–32B)上,GAMMA优于固定精度基线(最高+12.99平均得分)和基于搜索的混合精度方法(最高+7.00平均得分),并可在2.5比特平均精度下匹配固定3比特精度质量,显著降低内存开销。

原文摘要 · Abstract (English)

Mixed-precision quantization improves the budget--accuracy trade-off for large language models (LLMs) by allocating more bits to sensitive modules. However, automating this allocation at LLM scale faces a unique combination of constraints: learnable approaches require quantization-aware training, which is infeasible for billion-parameter models; training-free alternatives rely on static proxy metrics that miss cross-module interactions and must be recomputed per target budget; and search-based methods are expensive without guaranteeing exact budget compliance. We propose GAMMA, a quantizer-agnostic framework that learns module-wise precision preferences entirely within a post-training pipeline. GAMMA optimizes a teacher-forced hidden-state reconstruction objective under an augmented Lagrangian constraint, and projects the learned preferences into exact budget-feasible discrete assignments via integer programming. A key property is score reuse: because the learned preferences encode a stable sensitivity ranking rather than budget-specific weights, a single training run serves arbitrary deployment targets by re-solving only the integer program, reducing per-budget adaptation from hours to a few minutes. Across Llama and Qwen models (8B--32B), GAMMA outperforms both fixed-precision baselines (up to +12.99 Avg.) and search-based mixed-precision methods (up to +7.00 Avg.), and can match fixed 3-bit quality at 2.5-bit average precision, enabling deployment at substantially smaller memory footprints.

混合精度量化大模型整数规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。