arXiv:2601.11200cs.LGcs.AI2026-01ACL

用同家族大模型生成校准数据,显著降低量化误差。

FAQ: Mitigating Quantization Error via Regenerating Calibration Data with Family-Aware Quantization

  • 利用同家族大模型重生成高保真校准数据
  • 在多个模型上使精度损失减少最多28.5%
  • 适合部署大语言模型的资源受限场景

后训练量化(PTQ)虽能高效压缩大语言模型以适配资源受限设备,但校准数据的代表性与普适性仍是决定量化参数准确性的核心瓶颈。传统方法依赖有限样本,难以捕捉推理阶段的激活分布,导致量化参数偏差。为此,我们提出家族感知量化(FAQ),一种通过同家族大模型生成高保真校准数据的框架。具体而言,将原始校准样本输入同家族更大模型,利用其高度一致的知识体系重生成一系列高质量校准数据。随后,在专家指导下通过群体竞争筛选最优样本,并重新归一化以增强标准PTQ效果。在Qwen3-8B等多系列模型上的实验表明,相比使用原始校准数据的基线,FAQ可将精度损失降低最高达28.5%,展现出强大潜力。

原文摘要 · Abstract (English)

Although post-training quantization (PTQ) provides an efficient numerical compression scheme for deploying large language models (LLMs) on resource-constrained devices, the representativeness and universality of calibration data remain a core bottleneck in determining the accuracy of quantization parameters. Traditional PTQ methods typically rely on limited samples, making it difficult to capture the activation distribution during the inference phase, leading to biases in quantization parameters. To address this, we propose \textbf{FAQ} (Family-Aware Quantization), a calibration data regeneration framework that leverages prior knowledge from LLMs of the same family to generate high-fidelity calibration samples. Specifically, FAQ first inputs the original calibration samples into a larger LLM from the same family as the target model, regenerating a series of high-fidelity calibration data using a highly consistent knowledge system. Subsequently, this data, carrying Chain-of-Thought reasoning and conforming to the expected activation distribution, undergoes group competition under expert guidance to select the best samples, which are then re-normalized to enhance the effectiveness of standard PTQ. Experiments on multiple model series, including Qwen3-8B, show that FAQ reduces accuracy loss by up to 28.5\% compared to the baseline with original calibration data, demonstrating its powerful potential and contribution.

量化大模型校准数据PTQ

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。