arXiv:2603.16331cs.LG2026-03

发现大模型有隐藏纠错能力,无需修改文本也能自我修正错误。

Decoding the Critique Mechanism in Large Reasoning Models

  • 通过插入算术错误测试模型纠错机制
  • 即使错误传播全程,仍能得出正确答案
  • 发现可解释的纠错向量,提升推理性能

大型推理模型(LRMs)具备回溯和自验证机制,可修正中间步骤并取得良好逻辑任务表现。我们假设此类行为仅在模型具备足够强的‘批判’能力时才有效。本文系统研究当前LRMs如何通过在推理链中插入算术错误来恢复。值得注意的是,尽管错误在整个思维链中持续传播且未被显式修正,模型在思考结束后仍能得出正确答案。这表明存在一种内部机制帮助模型检测错误并触发自纠正,我们称之为‘隐藏批判能力’。基于特征空间分析,我们识别出一个高度可解释的‘批判向量’代表该行为。跨多个模型规模与家族的实验证明,用此向量引导潜在表示可提升错误检测能力,并在不增加训练成本的情况下增强测试时扩展性能。研究为理解LRMs的批判行为提供了新视角,指出了控制与改进其自验证机制的可行方向。代码已开源。

原文摘要 · Abstract (English)

Large Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions, yielding strong performance on complex logical benchmarks. We hypothesize that such behaviors are beneficial only when the model has sufficiently strong ``critique'' ability to detect its own mistakes. This work systematically investigates how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate reasoning steps. Notably, we discover a peculiar yet important phenomenon: despite the error propagating throughout the entire chain-of-thought (CoT) without any verbalized correction, the model still reaches the correct final answer after the thinking process finishes. This recovery implies the existence of an internal mechanism helping the model to detect errors and trigger self-correction, which we refer to as the \textit{hidden critique ability}. Building on feature space analysis, we identify a highly interpretable \textit{critique vector} representing this behavior. Extensive experiments across multiple model scales and families demonstrate that steering latent representations with this vector improves the model's error detection capability and enhances the performance of test-time scaling at no extra training cost. Our findings provide a valuable understanding of LRMs' critique behavior, suggesting a promising direction to control and improve their self-verification mechanism. Our code is available at: https://github.com/mail-research/lrm-critique-vectors.

大模型推理自验证纠错机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。