用强化学习让大模型学会为不同检索器定制查询策略。
Understanding the Behaviors of Environment-aware Information Retrieval
- 通过强化学习让大模型适配不同检索器的查询风格。
- 不同检索器最优查询风格差异显著，迁移效果差。
- 引入人类指导和模型规模扩展可提升性能。
最近的检索增强生成（RAG）方法在处理复杂查询方面表现出强大能力，但现有研究忽视了一个关键挑战：不同检索器需要根本不同的查询构建策略才能达到最佳性能。本文首次系统分析了大语言模型如何通过强化学习（RL）适应不同检索器的查询策略。实证研究表明，强化学习能有效使大模型根据特定检索器特性定制查询。我们发现，不同检索器展现出出人意料的差异性最优查询风格（如描述型与疑问型），表明为某一检索器学习的策略对其他检索器无效。此外，通过引入检索器特异性的人类指导并扩大模型规模，可进一步提升性能。为支持多步检索轨迹的学习，我们提出基于分叉的滚动策略，显著改善训练稳定性。本工作提供了首个实证证据与可操作洞察，推动真正具备检索器感知能力的RAG系统建设。代码与资源见https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval。
原文摘要 · Abstract (English)
Recent retrieval-augmented generation (RAG) approaches have demonstrated strong capability in handling complex queries, yet current research overlooks a critical challenge: different retrievers require fundamentally different query formulation strategies for optimal performance. In this work, we present the first systematic analysis of how LLMs can learn to adapt their query formulation strategies for different retrievers via reinforcement learning (RL). Our empirical study reveals that RL effectively teaches an LLM to tailor its queries to specific retriever characteristics. We discover that different retrievers exhibit surprisingly distinct optimal query styles (e.g., descriptive vs. question-like), suggesting strategies learned for one retriever ineffective for another. We further show that performance can be enhanced by incorporating retriever-specific human guidance and by scaling model size. To facilitate learning over multi-retrieval-step trajectories, we introduce a branching-based rollout technique that improves training stability. Our work provides the first empirical evidence and actionable insights for building truly retriever-aware RAG systems. Code and resources are available at https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。