分析1400万次真实搜索,发现智能搜索行为有迹可循。
Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests
- 用大模型标注海量搜索日志,识别任务意图与查询演变
- 超90%多轮搜索不超过10步,近90%步骤间隔少于1分钟
- 新关键词约54%源自之前检索结果,适合优化搜索策略
基于来自DeepResearchGym的1444万条搜索请求(397万次会话),本研究对大语言模型驱动的搜索代理进行了大规模日志分析。通过会话分割与大模型标注,我们定义了会话级意图和步骤级查询改写标签,并提出上下文驱动的术语采纳率(CTAR)来量化新引入关键词是否可追溯至先前检索证据。分析显示:超过90%的多轮会话在十步内完成,89%的步骤间隔低于一分钟;事实类搜索重复率随时间上升,而推理类搜索持续广泛探索;多数查询改写可追溯至检索证据,平均54%的新词出现在累积证据中,且部分可关联更早步骤。这些发现为重复感知终止、意图自适应检索预算分配及跨步骤上下文追踪提供了候选信号。数据集已公开于HuggingFace。
原文摘要 · Abstract (English)
LLM-powered search agents are increasingly being used for multi-step information seeking tasks, yet the IR community lacks empirical understanding of how agentic search sessions unfold and how retrieved evidence is reflected in later queries. This paper presents a large-scale log analysis of agentic search based on 14.44M search requests (3.97M sessions) collected from DeepResearchGym, i.e., an open-source search API accessed by external agentic clients. We sessionize the logs, assign session-level intents and step-wise query-reformulation labels using LLM-based annotation, and propose Context-driven Term Adoption Rate (CTAR) to quantify whether newly introduced query terms are lexically traceable to previously retrieved evidence. Our analyses reveal distinctive behavioral patterns. First, over 90\% of multi-turn sessions contain at most ten steps, and 89\% of inter-step intervals fall under one minute. Second, behavior varies by intent. Fact-seeking sessions exhibit high repetition that increases over time, while sessions requiring reasoning sustain broader exploration. Third, query reformulations are often traceable to retrieved evidence across steps. On average, 54\% of newly introduced query terms appear in the accumulated evidence context, with additional traceability to earlier steps beyond the most recent retrieval. These findings provide candidate signals for repetition-aware stopping, intent-adaptive retrieval budgeting, and explicit cross-step context tracking. We released the anonymized logs, making them available at a public HuggingFace~\chref{https://huggingface.co/datasets/cx-cmu/deepresearchgym-agentic-search-logs}{repository}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。