为大模型长程推理设计自动纠错系统，显著提升复杂任务成功率。
ReFlect: An Effective Harness System for Complex Long-Horizon LLM Reasoning
- 在模型推理外添加确定性纠错包装器，实现错误检测与恢复。
- 在6个任务上将成功率从41%提升至56%，最高增益达29个百分点。
- 无需训练、不依赖模型规模，适合各类大模型的推理增强。
当前大模型推理范式如思维链、ReAct和事后自我批判，依赖两个在长程多阶段任务中失效的假设。错误在推理步骤间无声累积，引发关键问题：能否构建有效系统检测并修复自身失败？本文提出ReFlect，一种面向大模型推理的独立纠错框架，以确定性包装器形式集成错误检测与恢复逻辑。在6个推理领域控制实验显示，提示级自我批判生成公式化模板，在100个审查块中有90个未发现任何问题，且模型在至少76%情况下错误接受错误答案。ReFlect在六种模型上取得41%（gpt-4o-mini）到56%（Claude Sonnet 4.5）的任务成功率，相比直接思维链（Direct CoT），每模型提升7个百分点（Qwen2.5-72B）至29个百分点（Claude Sonnet 4.5）。同时，将SWE-bench补丁结构质量从0%提升至82%（Qwen2.5-72B）至87%（GPT-4o）。值得注意的是，纠错增益与模型基线成功率呈负相关（斜率-1.69，相关系数r=-0.76）：每损失1个百分点，可机械恢复1.69个百分点。在Llama-3.3-70B和Qwen2.5-72B上，仅引入结构化推理状态与操作符，提升仅为15.0%–18.7%。该系统模型无关、免训练，完全在推理时运行。
原文摘要 · Abstract (English)
Current reasoning paradigms for LLMs include chain-of-thought, ReAct, and post-hoc self-critique. These paradigms rely on two assumptions that fail on long-horizon, multi-stage tasks. As a result, errors accumulate silently across reasoning steps, leaving an open question: can a reasoning system effectively detect and recover from its own failures? We present ReFlect, a \emph{harness} system for LLM reasoning that creates standalone error detection and recovery logic as a deterministic wrapper around the model. Controlled experiments across 6 reasoning domains show that prompt-level self-critique produces formulaic templates that flag no issues in 90 of 100 audited reflection blocks, and the investigated LLMs wrongly accept a wrong answer in at least 76\% of cases. Our ReFlect harness achieves task success rates ranging from 41\% on gpt-4o-mini to 56\% on Claude Sonnet 4.5 across six models spanning small and frontier scale, with per-model gains over Direct CoT ranging from +7 pp on Qwen2.5-72B to +29 pp on Claude Sonnet 4.5, and additionally raises SWE-bench patch-structural quality from 0\% (Direct CoT) to between 82\% (Qwen2.5-72B) and 87\% (GPT-4o). Notably, the harness gain is inversely proportional to the model's Direct CoT task success rate (the fitted slope is -1.69 with r=-0.76): each pp lost in baseline success rate is mechanically recovered by 1.69 pp of harness gain. We spot that adding structured reasoning state and operators yields only 15.0--18.7\% pair-mean on Llama-3.3-70B and Qwen2.5-72B because models at this scale cannot reliably populate the state its operators require. ReFlect is model-agnostic, training-free, and operates entirely at inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。