arXiv:2605.17997cs.LGcs.AI2026-05

通过模块自适应调整残差强度,提升低比特量化精度。

MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization

论文配图:MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization
图 1 · 摘自论文原文
  • 为每个模块动态分配残差缩放系数,平衡误差修正与偏差抑制。
  • 4比特以下量化下,大语言模型性能提升最高达20.2%。
  • 基于PID反馈机制自动优化系数,无需逐模块搜索。

近期基于残差重构的低比特后训练量化方法通过引入跨层残差,有效缓解了前层累积误差问题。然而,这类残差可能因重构方法依赖的海森近似(Hessian-approximation, HA)假设引入额外偏差,导致量化性能不佳。本文分析发现,对残差项乘以缩放系数可直接缓解由残差强度引发的HA偏差,同时保留累积误差修正能力。更重要的是,该权衡关系具有模块依赖性,单一全局缩放系数难以兼顾各模块需求。为此,提出模块自适应残差重构(MARR),为每个模块分配独立缩放系数,实现误差修正与残留偏差间的自适应平衡。为避免昂贵的逐模块系数搜索并获得稳定估计,设计基于比例-积分-微分(PID)的自适应更新策略,以重构误差为反馈逐步优化系数。在多个典型大语言模型(LLMs)和视觉变换器(ViTs)上实验表明,MARR在低比特量化(≤4比特)下显著优于现有残差重构方法,大语言模型性能最高提升20.2%,视觉变换器相对提升最高达4.6%。代码将在论文接受后公开。

原文摘要 · Abstract (English)

Recently, residual reconstruction-based model quantization methods have achieved promising performance in low-bit post-training quantization (PTQ) by introducing cross-layer residuals to reduce error accumulated from previous layers.However, these residuals may also introduce additional bias arising from the Hessian-approximation (HA) assumption underlying reconstruction-based PTQ, leading to suboptimal quantization performance.In this work, we analyze that multiplying the residual term by a scaling coefficient provides a direct way to mitigate the HA bias associated with residual strength, while preserving accumulated-error correction. More importantly, we observe that this trade-off is module-dependent, making a single global residual strength insufficient to balance effective correction and residual-related bias across modules.Based on these observations, we propose Module-Adaptive Residual Reconstruction (MARR), which assigns a module-specific scaling coefficient to adaptively balance accumulated-error correction and residual-related HA bias for each module.To avoid expensive per-module coefficient search and obtain a stable coefficient estimate, we design a Proportional-Integral-Derivative (PID)-based adaptive update strategy that uses reconstruction error as feedback to progressively refine this coefficient. Experiments on several typical large language models (LLMs) and vision transformers (ViTs) demonstrate the effectiveness of MARR under low-bit quantization (less than or equal to 4-bit), achieving up to 20.2% performance gains on LLMs and up to 4.6% relative gains on ViTs over the residual reconstruction state-of-the-art methods.Code will be made publicly available upon acceptance.

量化残差重构大模型自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。