让大模型学会失败后重试，提升检索能力。
ReZero: Enhancing LLM search ability by trying one-more-time
- 用强化学习直接奖励模型失败后重试，鼓励持续探索
- 在知识密集型任务上准确率提升至46.88%，远超25%基线
- 适合需要多轮尝试的复杂信息检索场景
检索增强生成（RAG）能提升大语言模型在知识密集型任务上的表现，但其效果高度依赖初始检索查询的质量。现有方法通常使用强化学习优化查询生成或结果推理，却未显式鼓励在首次搜索失败后继续尝试。本文提出ReZero（Retry-Zero），一种新型强化学习框架，通过直接奖励模型在初次失败后重试查询的行为，激励其探索替代查询而非过早终止。实验表明，ReZero显著提升性能，在测试中达到46.88%的准确率，远高于25%的基线水平。该方法通过奖励坚持，增强了大模型在初始查询不足时的鲁棒性，适用于复杂信息检索场景。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) improves Large Language Model (LLM) performance on knowledge-intensive tasks but depends heavily on initial search query quality. Current methods, often using Reinforcement Learning (RL), typically focus on query formulation or reasoning over results, without explicitly encouraging persistence after a failed search. We introduce ReZero (Retry-Zero), a novel RL framework that directly rewards the act of retrying a search query following an initial unsuccessful attempt. This incentivizes the LLM to explore alternative queries rather than prematurely halting. ReZero demonstrates significant improvement, achieving 46.88% accuracy compared to a 25% baseline. By rewarding persistence, ReZero enhances LLM robustness in complex information-seeking scenarios where initial queries may prove insufficient.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。