arXiv:2509.06297cs.LG2025-09被引 5

提出层间输出近似量化,让模型压缩更贴近原始输出。

LoaQ: Layer-wise Output Approximation Quantization

  • 在逐层量化中引入输出匹配因子,优化层间一致性
  • 支持权重和激活联合量化,显著提升压缩后模型性能
  • 方法简洁可嵌入现有流程,适合大规模语言模型部署

模型量化中一个自然的想法是让每个组件的量化输出尽可能接近原始输出。然而,现有逐层后训练量化(PTQ)方法主要关注线性层权重的近似,导致局部目标与整体输出对齐不足。受主流大模型结构启发,本文提出LoaQ,通过在逐层量化框架中引入输出匹配因子来优化线性层输出,使量化更符合原始设计意图。该方法具备闭式解,与现有技术正交,可无缝集成至现有量化流程。在LLaMA与Qwen模型族上的实验表明,无论仅量化权重还是联合量化权重与激活,LoaQ均表现优异,显著提升量化质量,具有推动后训练量化前沿的潜力。

原文摘要 · Abstract (English)

A natural and intuitive idea in model quantization is to approximate each component's quantized output to match its original. Motivated by this idea, most layer-wise post-training quantization (PTQ) methods focus on weight approximation at the linear-layer level. As a result, this local objective often yields insufficient approximations and practical deviations from the guiding intuition. Recent work has improved the approximation of linear-layer outputs within the layer-wise PTQ framework, but such refinements remain inadequate for achieving alignment with the full-model output. Based on a deeper understanding of the structure of mainstream LLMs, we propose LoaQ, which incorporates output-matching factors when quantizing linear layers within the layer-wise PTQ framework. It better aligns with this intuition and can feature a simple closed-form solution, making it orthogonal to existing techniques and readily integrable into existing quantization pipelines. Experiments on the LLaMA and Qwen model families demonstrate that LoaQ performs effectively in both weight-only and weight-activation quantization. By integrating seamlessly with existing quantization strategies, it further enhances overall quantization quality and shows strong potential to advance the frontier of post-training quantization.

模型量化大模型压缩后训练量化输出对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。