arXiv:2607.09999cs.CLcs.LG2026-07

量化后模型推理过程悄悄出错,准确率却不变,难被发现。

Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts

  • 用六类错误分类法分析3万条推理输出,发现精度下降时推理路径异常
  • 小模型在NF4下正确答案率下降,大模型(12B以上)保持稳定
  • 传统评估无法发现隐蔽错误,适合关注模型可靠性的研究者看

我们发现,后训练量化可在任务准确率不变的情况下悄然改变大语言模型的推理方式。基于两名独立人工标注者验证的六类失败分类法(Cohen's κ = 0.906),我们对五个指令微调的LLM(3B–14B参数)在三种量化精度(FP32、FP16、NF4)下,于四个推理基准上的3万条思维链输出进行分类。尽管准确率在各精度间稳健(最高下降3.1个百分点),但‘空心收敛’(正确答案通过不完整或不可验证的推理得出)在NF4下呈现显著的规模依赖性变化:两个最小模型明显下降,而12B及以上模型保持不变。该现象还具有基准特异性:GSM8K完全免疫,而LogiQA和ARC-Challenge表现最剧烈。此外,在LLaMA 3.2-3B中,快捷路径崩溃从44%升至78%,而信心雪球效应从15.8%降至接近零,这种质变无法被准确率捕捉。最后,我们证明仅靠表面文本特征无法可靠检测空心收敛(最佳F1=0.53),表明其是标准评估流程难以发现的部署级风险。

原文摘要 · Abstract (English)

We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved. Using a six-category failure taxonomy validated by two independent human annotators (Cohen's $κ$ = 0.906), we classify 30,000 chain-of-thought outputs from five instruction-tuned LLMs (3B--14B parameters) across three quantization precisions (FP32, FP16, NF4) and four reasoning benchmarks. We find that while accuracy is robust across precisions (maximum 3.1 pp drop), Hollow Convergence (correct answers reached through incomplete or unverifiable reasoning) shows a significant size-dependent shift under NF4, dropping sharply for the two smallest models tested but remaining invariant for models at 12B parameters and above. This effect is also benchmark-specific: GSM8K is categorically immune while LogiQA and ARC-Challenge show the largest shifts. Furthermore, under NF4, Shortcut Collapse rises from 44% to 78% of wrong-answer failures in LLaMA 3.2-3B while Confidence Snowballing collapses from 15.8% to near zero, a qualitative shift invisible to accuracy metrics. Finally, we show Hollow Convergence cannot be reliably detected from surface-level text features (best F1 = 0.53), establishing it as a deployment-relevant failure mode that standard evaluation pipelines cannot catch.

量化推理安全模型可靠性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。