arXiv:2505.02811cs.AIcs.CL2025-05被引 32

让AI学会判断何时该继续搜索,避免盲目查资料或过早下结论。

Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing

  • 通过让AI自己模拟多轮检索过程生成训练数据,提升自我评估能力。
  • 在多个基准测试中显著减少错误答案,提升检索准确率。
  • 无需人工标注中间步骤,适合希望优化现有RAG系统的开发者。

检索增强生成(RAG)在提升语言模型知识和减少幻觉方面表现优异,但复杂任务所需的多轮检索仍具挑战性,现有系统常因缺乏自我怀疑意识而过度搜索或过早回答。当前方法或需大量人工标注的中间过程数据,或性能不佳。本文提出SIM-RAG框架,通过让RAG系统自实践多轮检索,生成带中间推理步骤的合成数据,构建轻量级信息充分性判别器(Critic)。该Critic在推理时评估每轮是否已获取足够信息,指导检索决策,实现上下文内强化学习。实验表明,SIM-RAG在多个主流RAG基准上表现优异,系统高效且无需修改现有LLM或搜索引擎,数据高效,避免昂贵的人工标注。

原文摘要 · Abstract (English)

Retrieval Augmented Generation (RAG) has shown strong capability in enhancing language models' knowledge and reducing AI generative hallucinations, driving its widespread use. However, complex tasks requiring multi-round retrieval remain challenging, and early attempts tend to be overly optimistic without a good sense of self-skepticism. Current multi-round RAG systems may continue searching even when enough information has already been retrieved, or they may provide incorrect answers without having sufficient information or knowledge. Existing solutions either require large amounts of expensive human-labeled process supervision data or lead to subpar performance. This paper aims to address these limitations by introducing a new framework, SIM-RAG, to explicitly enhance RAG systems' self-awareness and multi-round retrieval capabilities. To train SIM-RAG, we first let a RAG system self-practice multi-round retrieval, augmenting existing question-answer pairs with intermediate inner monologue reasoning steps to generate synthetic training data. For each pair, the system may explore multiple retrieval paths, which are labeled as successful if they reach the correct answer and unsuccessful otherwise. Using this data, we train a lightweight information sufficiency Critic. At inference time, the Critic evaluates whether the RAG system has retrieved sufficient information at each round, guiding retrieval decisions and improving system-level self-awareness through in-context reinforcement learning. Experiments across multiple prominent RAG benchmarks show that SIM-RAG is an effective multi-round RAG solution. Furthermore, this framework is system-efficient, adding a lightweight component to RAG without requiring modifications to existing LLMs or search engines, and data-efficient, eliminating the need for costly human-annotated mid-step retrieval process supervision data.

RAG多轮检索自我评估轻量模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。