arXiv:2608.01913cs.AIcs.CL2026-08

分析长时序搜索代理的搜索行为,发现搜得越多不等于答得越好。

Diagnosing Search Behavior and Failure Modes in Long-Horizon Search Agents

  • 通过分阶段诊断,区分证据获取与使用效果
  • 高质量证据早出现,但多数代理仍盲目持续搜索
  • 适合改进搜索系统设计,尤其在查询优化和停止策略

深度搜索代理通过迭代查询获取支持性证据来回答复杂信息问题,但搜索努力与答案质量的关系尚不明确。本文通过轨迹级诊断研究长时序搜索代理的行为。利用人工标注的文档相关性判断,评估每一步检索到的证据,并分离出两个阶段:代理获取了什么证据,以及如何有效利用这些证据。这一区分使我们能将失败分解为两类:检索空白(必要证据未找到)与利用空白(相关证据被获取但未正确使用)。在固定检索模型和评估框架下,对比了六种代理在BrowseComp-Plus上的表现,并在BrowseComp上通过开放网络搜索接口验证结果。跨设置发现,搜索次数与答案准确率仅弱相关;答案质量更依赖于检索证据的整体质量,尤其是累计检索召回率,而非搜索次数或上下文消耗量。有用证据常在早期出现,但代理倾向于继续搜索,导致大量低效步骤。在查询层面,探索性改写仍有价值,但表现最佳的代理发出的冗余查询远少。本研究系统刻画了长时序搜索代理的搜索行为与失效模式,为构建更优深度研究系统指明方向,包括更强的查询生成、更有效的证据筛选与上下文管理,以及基于是否已获足够支持证据的停止机制。

原文摘要 · Abstract (English)

Deep search agents answer difficult information-seeking questions by iteratively issuing search queries to gather supporting evidence, but it remains unclear whether and how greater search effort leads to better answers. We study these questions through a trajectory-level diagnosis of long-horizon search agents. Using human-annotated document-level relevance judgments, we evaluate the evidence retrieved at each search step and separate two stages of agent behavior: what evidence an agent retrieves and how effectively it uses that evidence. This distinction further allows us to decompose failures into retrieval gaps, where the necessary evidence is never found, and utilization gaps, where relevant evidence is retrieved but not used correctly. With the retrieval model and evaluation harness held fixed, we compare six agents on BrowseComp-Plus and further validate our findings on BrowseComp with an open-web search API. Across settings, we find that search effort and answer quality are only weakly aligned. Answer accuracy is better correlated with the quality of retrieved evidence, especially cumulative retrieval recall, than with the number of searches or the amount of context consumed. Useful evidence often appears early in the trajectory, yet agents tend to continue searching, producing a long tail of low-yield retrieval steps. At the query level, exploratory reformulations remain useful, but the best-performing agents issue far fewer redundant queries. Overall, by systematically characterizing the search behavior and failure modes of long-horizon search agents, this work points to practical directions for building better deep research systems, including stronger query formulation, more effective evidence selection and context management, and stopping criteria based on whether sufficient supporting evidence has been retrieved.

搜索代理长时序任务失败分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。