arXiv:2601.06786cs.CL2026-01ACL被引 1

让大模型学会识别自己不知道的内容,提升推理准确性和可信度。

EpiCaR: Knowing What You Don't Know Matters for Better Reasoning in LLMs

  • 引入自评估信号,训练模型同时掌握推理能力与自我怀疑能力。
  • 在30亿以上参数模型上,准确率和校准度均优于基准方法。
  • 可减少3倍推理计算量,适合需要高可靠性推理的场景。

提升大语言模型(LLMs)的推理能力主要依赖于迭代式自训练,但现有方法仅强化成功推理路径,导致模型过度自信、失去不确定性表达能力,引发对齐中的模型坍塌问题。本文将推理训练重构为认知学习任务,要求模型不仅会推理,还要知道何时应信任自身推理。提出一种认知校准推理(EpiCaR)训练目标,在迭代监督微调框架中引入显式自评估信号,联合优化推理性能与校准性。在Llama-3与Qwen-3系列模型上的实验表明,该方法在准确率与校准性上均实现帕累托最优,尤其在30亿及以上参数模型中表现显著。该框架在外部分布数学推理(GSM8K)与代码生成(MBPP)任务上具有良好泛化能力。最终,具备足够推理能力的模型仅需10次采样即可达到标准自训练(STaR)30次采样的性能,推理计算量降低3倍。

原文摘要 · Abstract (English)

Improving the reasoning abilities of large language models (LLMs) has largely relied on iterative self-training with model-generated data. While effective at boosting accuracy, existing approaches primarily reinforce successful reasoning paths, incurring a substantial calibration cost: models become overconfident and lose the ability to represent uncertainty. This failure has been characterized as a form of model collapse in alignment, where predictive distributions degenerate toward low-variance point estimates. We address this issue by reframing reasoning training as an epistemic learning problem, in which models must learn not only how to reason, but also when their reasoning should be trusted. We propose epistemically-calibrated reasoning (EpiCaR) as a training objective that jointly optimizes reasoning performance and calibration, and instantiate it within an iterative supervised fine-tuning framework using explicit self-evaluation signals. Experiments on Llama-3 and Qwen-3 families demonstrate that our approach achieves Pareto-superiority over standard baselines in both accuracy and calibration, particularly in models with sufficient reasoning capacity (e.g., 3B+). This framework generalizes effectively to OOD mathematical reasoning (GSM8K) and code generation (MBPP). Ultimately, our approach enables a 3X reduction in inference compute, matching the K=30 performance of STaR with only K=10 samples in capable models.

大模型推理认知校准自评估高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。