发现大模型纠错能力存在盲区,外错能改而自错难纠。
Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
- 通过内外错误对照实验,分离出纠错失败是因能力不足还是未激活。
- 14个模型中64.5%存在自我纠错盲点,相同错误外部时可纠正,自生成时却不行。
- 添加'Wait'提示即可减少89.3%盲点,揭示纠错机制可被简单触发。
尽管大语言模型(LLMs)已推动AI发展,但依然会犯错并陷入无效推理路径。自我纠错对安全关键应用至关重要,但研究需区分激活失败与知识不足:当模型无法纠错时,是不能还是不?我们提出Self-Correction Bench,一个受控评估框架,通过将相同错误以外部(用户归因)或内部(模型归因)形式注入,在其他上下文完全一致的条件下进行测试。在14个开源非推理类模型上测试显示,存在64.5%的自我纠错盲点:模型可纠正外部错误,却无法纠正相同的内部错误,证明纠错能力存在但未被激活。对于模型自身生成的错误,其未能察觉的部分,在外部呈现时却能被识别。根源在于后训练数据构成:监督微调数据集缺乏纠错序列;仅用5,306条此类样本微调,即能使盲点降低76.0%。机制上,我们发现表示空间中一个可迁移的对话角色方向,因果性地控制自我纠错。添加'Wait'提示无需训练即可使盲点下降89.3%,且作用路径近乎独立,表明纠错激活不可简化为单一机制。
原文摘要 · Abstract (English)
Although large language models (LLMs) have transformed AI, they still make errors and follow unproductive reasoning paths. Self-correction is vital for safety-critical applications, but studying it requires disentangling activation failure from knowledge deficiency: when a model fails to correct an error, is it because it cannot, or because it does not? We introduce Self-Correction Bench, a controlled evaluation framework that isolates this distinction by injecting the same error as either an external (user-attributed) or internal (model-attributed) error, keeping all other context identical. Testing 14 open-source non-reasoning models reveals a 64.5% Self-Correction Blind Spot: models correct external errors but fail on identical internal ones, proving the capability exists but is not activated. On models' own naturally generated errors, a measurable share of what a model fails to catch in its own output is caught when the identical error is presented externally. We trace the cause to post-training data composition: supervised fine-tuning datasets lack error-correction sequences, and fine-tuning with as few as 5,306 such traces already reduces the blind spot by 76.0%. Mechanistically, we identify a transferable conversational-role direction in representation space that causally gates self-correction. Appending "Wait" requires no training yet reduces the blind spot by 89.3%, and operates through a nearly independent pathway, indicating that correction activation is not reducible to this single mechanism.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。