让大模型自我修正,提升推理准确率。
Recursive Think-Answer Process for LLMs and VLMs
- 引入递归思考-作答机制,通过多次迭代优化答案
- 使用置信度生成器引导改进,显著减少错误输出
- 适用于大语言模型和视觉语言模型,推理更稳定高效
Think-Answer 类推理器如 DeepSeek-R1 已通过可解释的内部推理取得显著进展。然而,尽管频繁出现类似 'Oops!' 的自我反思提示,其单次推理仍易出错。为此,我们提出一种高效的递归思考-作答过程(R-TAP),使模型能够进行迭代推理循环,生成更准确的答案,突破传统单次推理的局限。该方法的核心是置信度生成器,用于评估模型响应的确定性并指导后续优化。通过引入两种互补奖励——递归置信度提升奖励与最终答案置信度奖励,我们证明 R-TAP 增强后的模型在大语言模型(LLMs)和视觉语言模型(VLMs)上均持续优于传统单次推理方法。此外,分析模型响应中 'Oops!' 类表达的频率发现,应用 R-TAP 的模型显著减少了自我反思模式,实现更稳定、更快的推理时推理。我们希望 R-TAP 能为未来 AI 推理过程的高效化与精细化提供新方向。
原文摘要 · Abstract (English)
Think-Answer reasoners such as DeepSeek-R1 have made notable progress by leveraging interpretable internal reasoning. However, despite the frequent presence of self-reflective cues like "Oops!", they remain vulnerable to output errors during single-pass inference. To address this limitation, we propose an efficient Recursive Think-Answer Process (R-TAP) that enables models to engage in iterative reasoning cycles and generate more accurate answers, going beyond conventional single-pass approaches. Central to this approach is a confidence generator that evaluates the certainty of model responses and guides subsequent improvements. By incorporating two complementary rewards-Recursively Confidence Increase Reward and Final Answer Confidence Reward-we show that R-TAP-enhanced models consistently outperform conventional single-pass methods for both large language models (LLMs) and vision-language models (VLMs). Moreover, by analyzing the frequency of "Oops"-like expressions in model responses, we find that R-TAP-applied models exhibit significantly fewer self-reflective patterns, resulting in more stable and faster inference-time reasoning. We hope R-TAP pave the way evolving into efficient and elaborated methods to refine the reasoning processes of future AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。