用强化学习让对话搜索更智能,能随用户意图变化动态调整策略。
Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning
- 通过强化学习在多轮对话中交替执行搜索与推理,动态适应用户意图变化。
- 在4个基准测试上超越多个强基线模型,显著提升搜索准确率。
- 适合需要复杂交互的智能搜索系统研发者或对话系统工程师使用。
大语言模型已成为人机交互的重要接口,支持通过自然多轮对话进行信息查询和任务协助。在多轮对话中,用户意图随上下文不断演化,需进行上下文理解、查询重写及检索与生成的动态协调。现有方法通常采用静态的重写-检索-生成流程,分别优化各环节,忽视了混合主动行为的联合优化。尽管近期深度搜索代理在单轮场景中展示了检索与生成联合优化的有效性,但难以处理多轮交互。本文提出一种对话代理,通过强化学习在多轮中交错执行搜索与推理,学习探索性和适应性行为,并设计针对演变用户目标的奖励机制。在四个常用对话基准上的实验结果表明,该方法优于多个现有强基线模型。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have become a popular interface for human-AI interaction, supporting information seeking and task assistance through natural, multi-turn dialogue. To respond to users within multi-turn dialogues, the context-dependent user intent evolves across interactions, requiring contextual interpretation, query reformulation, and dynamic coordination between retrieval and generation. Existing studies usually follow static rewrite, retrieve, and generate pipelines, which optimize different procedures separately and overlook the mixed-initiative action optimization simultaneously. Although the recent developments in deep search agents demonstrate the effectiveness in jointly optimizing retrieval and generation via reasoning, these approaches focus on single-turn scenarios, which might lack the ability to handle multi-turn interactions. We introduce a conversational agent that interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through reinforcement learning (RL) training with tailored rewards towards evolving user goals. The experimental results across four widely used conversational benchmarks demonstrate the effectiveness of our methods by surpassing several existing strong baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。