提出无需额外计算开销的分层前瞻量化方法，显著提升大模型长序列推理精度与速度。
LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss
- 采用分层前瞻损失，通过隐式跨层协同保持残差流一致性。
- 在Qwen3-4B上实现15.11%的Pass@1提升，推理速度达FP16的3.42倍。
- 适合追求高效高精度推理部署的开发者和研究者。
大型推理模型（LRMs）通过长自回归解码达到竞赛级数学与编码准确率，但每标记解码成本成为部署主要瓶颈。权重量化是加速标准手段，但现有端到端量化感知训练（E2E QAT）方法在长序列推理基准上出现准确率下降，尽管困惑度与短序列准确率保持良好。通过系统梯度方向分析，我们识别出两个关键因素：(i) E2E监督下软最大值费舍尔度量削弱了键值缓存保真度；(ii) 校准数据与部署分布之间的海森子空间不匹配。为此，我们提出分层前瞻量化（LAQuant），一种无在线变换开销的层间权重仅量化方法，结合推理域校准与单层前瞻损失，隐式实现跨层协同，保持下一层数残差流。在W3G128量化下，对Qwen3-4B，LAQuant相比ParoQuant提升AIME25 Pass@1达15.11个百分点（在匹配校准下优于ParoQuant++ 1.93个百分点），并在RTX A6000上实现3.42倍解码加速，优于ParoQuant的3.01倍。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) reach competition-level math and coding accuracy via long autoregressive decoding, making per-token decoding cost a primary deployment concern. Weight quantization is the standard tool for acceleration, but representative recipes -- including state-of-the-art end-to-end (E2E) QAT -- lose accuracy on long-decoding reasoning benchmarks despite preserving perplexity and short-decode accuracy. Through a systematic gradient-direction analysis, we identify two factors driving this gap: (i) KV-cache fidelity preservation under the QAT loss, which E2E supervision attenuates via the softmax Fisher metric; and (ii) Hessian-subspace alignment between calibration data and the deployment distribution. We propose LookAhead Quantization (LAQuant), a layer-wise weight-only QAT method that addresses both factors without online-transform overhead by combining reasoning-domain calibration with a one-layer lookahead loss whose implicit cross-layer co-adaptation preserves the next-layer residual stream. For Qwen3-4B under W3G128 quantization, LAQuant improves AIME25 Pass@1 over ParoQuant by 15.11pp (1.93pp over ParoQuant++ at matched calibration) while achieving a 3.42x decoding speedup over FP16 on RTX A6000, compared with ParoQuant's 3.01x.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。