低比特量化会严重损害大模型数学推理能力，但微调可快速恢复
Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
- 提出多维度评估框架，分析量化对推理能力的影响
- 仅用545个示例微调3分钟，就能恢复近全精度的推理性能
- 错误诊断准确率达98.9%，帮助定位量化导致的失效问题
大语言模型在复杂数学推理基准（如MATH）上取得显著进展，但其巨大的计算需求制约了实际部署。模型量化通过使用低精度和低比特表示，成为降低内存占用和计算成本的有效策略。本文系统评估了量化对数学推理任务的影响，结果表明，如AWQ和GPTQ等激进量化方法在Llama-3模型上引入高达32.39%的准确率下降（平均11.31%），尤其影响数值计算与推理规划。为此，我们提出结合定性能力分析与定量误差评估的多维评估框架，并开发针对性恢复策略：在仅545个特定任务样本上，用4块GPU微调3分钟，即可将量化模型的推理能力恢复至接近全精度水平。此外，我们的错误评估流水线在3,366个失败案例中实现98.9%的错误诊断与定位准确率，为缓解量化引起的性能退化提供可操作洞见。
原文摘要 · Abstract (English)
Large language models have achieved significant advancements in complex mathematical reasoning benchmarks, such as MATH. However, their substantial computational requirements present challenges for practical deployment. Model quantization has emerged as an effective strategy to reduce memory usage and computational costs by employing lower precision and bit-width representations. In this study, we systematically evaluate the impact of quantization on mathematical reasoning tasks. Our results demonstrate that aggressive quantization methods like AWQ and GPTQ introduce up to 32.39% accuracy degradation (average 11.31%) on Llama-3 models, particularly in numerical computation and reasoning planning. To address this, we introduce a multidimensional evaluation framework combining qualitative capability analysis and quantitative error assessment. We further develop targeted recovery strategies, showing that fine-tuning quantized models on only 545 task-specific examples for 3 minutes on 4 GPUs effectively restores reasoning capabilities to near full-precision levels. Additionally, our error assessment pipeline achieves 98.9% accuracy in diagnosing and localizing errors across 3,366 failure cases, providing actionable insights for mitigating quantization-induced degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。