用外部检索增强大模型的深度推理能力，让复杂问题解决更可靠。
RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
- 结合检索与树状搜索，让模型分步生成并验证推理路径。
- 在Llama-3.1和GPT-4o上，复杂推理准确率显著提升。
- 适合需要高可靠性推理的任务，如科学问答与逻辑推演。
现有大语言模型虽具强大问题求解能力，但在复杂推理任务中仍显不足。尽管思维链与树状搜索方法取得进展，但主要依赖模型内部知识，难以处理需多步推理的复杂任务。本文提出RAG-Star，一种融合外部检索信息的新型RAG方法，通过蒙特卡洛树搜索迭代生成中间子查询与答案，并利用查询与答案感知的奖励建模进行检索增强型验证，以反馈优化模型内在推理过程。实验表明，该方法在Llama-3.1-8B-Instruct和GPT-4o上显著优于现有RAG与推理方法。
原文摘要 · Abstract (English)
Existing large language models (LLMs) show exceptional problem-solving capabilities but might struggle with complex reasoning tasks. Despite the successes of chain-of-thought and tree-based search methods, they mainly depend on the internal knowledge of LLMs to search over intermediate reasoning steps, limited to dealing with simple tasks involving fewer reasoning steps. In this paper, we propose \textbf{RAG-Star}, a novel RAG approach that integrates the retrieved information to guide the tree-based deliberative reasoning process that relies on the inherent knowledge of LLMs. By leveraging Monte Carlo Tree Search, RAG-Star iteratively plans intermediate sub-queries and answers for reasoning based on the LLM itself. To consolidate internal and external knowledge, we propose an retrieval-augmented verification that utilizes query- and answer-aware reward modeling to provide feedback for the inherent reasoning of LLMs. Our experiments involving Llama-3.1-8B-Instruct and GPT-4o demonstrate that RAG-Star significantly outperforms previous RAG and reasoning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。