arXiv:2510.16062cs.CLcs.AI2025-10NeurIPS被引 12

测试大模型能否自我纠错,发现方法有效但效率低。

Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs

  • 设计CorrectBench基准,评估三种纠错策略。
  • 复杂推理任务中纠错提升准确率,混合策略更优。
  • 简单思维链基线表现接近最优,适合高效场景。

自纠正能力对提升大语言模型(LLM)的推理性能至关重要。尽管已有多种自纠正方法提出,但其综合评估仍不充分,且模型是否真正具备自我纠错能力仍是重要关注点。本文提出CorrectBench基准,用于评估内在、外部及微调类自纠正策略在常识推理、数学推理和代码生成三个任务上的表现。结果表明:1)自纠正方法能提升准确率,尤其在复杂推理任务中;2)融合多种策略可进一步提升性能,但降低效率;3)推理型模型(如DeepSeek-R1)在额外自纠正下优化有限且耗时高。有趣的是,较简单的思维链(CoT)基线展现出竞争力的准确率与效率。研究揭示了自纠正的潜力,也凸显了推理能力与效率平衡的挑战,呼吁未来研究聚焦于此。项目页面:https://correctbench.github.io/

原文摘要 · Abstract (English)

Self-correction of large language models (LLMs) emerges as a critical component for enhancing their reasoning performance. Although various self-correction methods have been proposed, a comprehensive evaluation of these methods remains largely unexplored, and the question of whether LLMs can truly correct themselves is a matter of significant interest and concern. In this study, we introduce CorrectBench, a benchmark developed to evaluate the effectiveness of self-correction strategies, including intrinsic, external, and fine-tuned approaches, across three tasks: commonsense reasoning, mathematical reasoning, and code generation. Our findings reveal that: 1) Self-correction methods can improve accuracy, especially for complex reasoning tasks; 2) Mixing different self-correction strategies yields further improvements, though it reduces efficiency; 3) Reasoning LLMs (e.g., DeepSeek-R1) have limited optimization under additional self-correction methods and have high time costs. Interestingly, a comparatively simple chain-of-thought (CoT) baseline demonstrates competitive accuracy and efficiency. These results underscore the potential of self-correction to enhance LLM's reasoning performance while highlighting the ongoing challenge of improving their efficiency. Consequently, we advocate for further research focused on optimizing the balance between reasoning capabilities and operational efficiency. Project Page: https://correctbench.github.io/

自纠正大模型推理评估效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。