arXiv:2411.14574cs.AI2024-11被引 1

针对复杂对话查询,设计低成本智能路由系统提升问答质量。

SRSA: A Cost-Efficient Strategy-Router Search Agent for Real-world Human-Machine Interactions

  • 根据查询特点动态选择搜索策略,实现精准路由。
  • 在真实对话数据集上,生成更全面、新颖且可操作的回答。
  • 无需微调大模型,适合实际人机交互场景应用。

随着大语言模型(LLMs)展现强大涌现能力并广泛应用,基于LLM的搜索代理研究迅速增长。在真实场景中,用户常输入上下文丰富、高度个性化的查询,对模型理解上下文和生成恰当回答提出挑战。然而,以往研究多未聚焦真实人机对话场景,且忽略响应质量与计算成本之间的平衡,强制所有查询采用相同处理流程。为此,我们提出策略路由搜索代理(SRSA),可根据不同查询动态选择合适搜索策略,支持细粒度串行搜索,在较低成本下获得高质量结果。为评估该方法,我们构建了新的上下文查询增强数据集(CQED),模拟人类与聊天机器人的真实日常交互。通过基于LLM的自动评估指标,从信息量、完整性、新颖性和可操作性等方面评估性能。结果表明,SRSA有效解决了长文本上下文查询导致的退化回答问题,能高效解析复杂查询,生成更全面、更具信息量的回复,且无需微调大模型。

原文摘要 · Abstract (English)

Recently, as Large Language Models (LLMs) have shown impressive emerging capabilities and gained widespread popularity, research on LLM-based search agents has proliferated. In real-world situations, users often input contextual and highly personalized queries to chatbots, challenging LLMs to capture context and generate appropriate answers. However, much of the prior research has not focused specifically on authentic human-machine dialogue scenarios. It also ignores the important balance between response quality and computational cost by forcing all queries to follow the same agent process. To address these gaps, we propose a Strategy-Router Search Agent (SRSA), routing different queries to appropriate search strategies and enabling fine-grained serial searches to obtain high-quality results at a relatively low cost. To evaluate our work, we introduce a new dataset, Contextual Query Enhancement Dataset (CQED), comprising contextual queries to simulate authentic and daily interactions between humans and chatbots. Using LLM-based automatic evaluation metrics, we assessed SRSA's performance in terms of informativeness, completeness, novelty, and actionability. To conclude, SRSA provides an approach that resolves the issue of simple serial searches leading to degenerate answers for lengthy and contextual queries, effectively and efficiently parses complex user queries, and generates more comprehensive and informative responses without fine-tuning an LLM.

搜索代理对话系统大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。