让大模型自己判断何时停止推理,提升效率和准确率。
CoRE: Enhancing Metacognition with Label-free Self-evaluation in LRMs
- 用隐藏状态轨迹检测冗余思考模式,无需外部标签。
- 减少13.7%到33.2%的推理步骤,准确率提升约10%。
- 适合追求高效推理的大模型应用者使用。
大型推理模型(LRMs)在数学和程序合成等领域表现出色,但常出现过度推理——即过多且重复的推理步骤,导致推理效率低下。如何让模型在无外部标签的情况下自主评估自身推理过程的正确性,成为关键问题。为此,本文提出链式推理嵌入(CoRE),利用隐空间中的隐藏状态轨迹实现对中间推理步骤的无标签自评估,从而增强模型的元认知能力,提升推理效率。通过分析CoRE轨迹的几何特性,发现冗余推理通常表现为循环波动,对应重复且无意识的反思与探索。基于此,我们进一步设计了无需训练、无标签的自评估框架CoRE-Eval,可动态识别此类模式并决定是否提前终止推理。在多个数学推理基准测试(GSM8K、MATH-500、AIME)及7B至32B不同规模模型上进行的大量实验表明,CoRE-Eval将推理链长度减少13.7%至33.2%,同时准确率提升约10%,32B模型在挑战性AIME基准上达到70.0%准确率。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) have demonstrated impressive capabilities in domains like mathematics and program synthesis. Despite their strong performance, LRMs often exhibit overthinking -- excessive and redundant reasoning steps that introduce inefficiencies during inference. This phenomenon raises an important question for LRM self-evaluation: How can a model autonomously assess the correctness of its own reasoning trajectory without external labels? To address this, we propose Chain-of-Reasoning Embedding (CoRE), a series of hidden states in latent space to enable label-free self-evaluation on intermediate reasoning steps of LRMs, so as to enhance metacognition abilities for improved reasoning efficiency. By analyzing the geometric properties of the CoRE trajectories, we reveal that redundant reasoning usually presents cyclical fluctuations, which correspond to repetitive and unconscious reflection/exploration. Leveraging this insight, we further introduce a training-free, label-free self-evaluation framework, CoRE-Eval, to detect such patterns and dynamically determine whether to terminate reasoning early. Extensive experiments on mathematical reasoning benchmarks (GSM8K, MATH-500, and AIME) and across model sizes from 7B to 32B demonstrate that CoRE-Eval reduces chain-of-thought length by 13.7% to 33.2% while improving answer accuracy by around 10%, achieving 70.0% accuracy on the challenging AIME benchmark with the 32B model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。