arXiv:2607.26070cs.IRcs.AI2026-07

构建可复现的离线维基搜索环境,解决智能体检索评估中的不可比问题。

SimpleWikiSearch: A Clean Offline Wikipedia Environment for Agentic Search

  • 从维基全量数据出发,明确定义清洗、切块、索引等全流程
  • 提供包含搜索/打开链接/提交答案的极简工具接口,支持基准测试
  • 适用于需公平对比开源与闭源模型的研究者

基于大语言模型的智能体搜索系统常被评估为仅依赖模型本身,但实际表现还受维基快照、预处理流程、分块策略、检索后端、工具接口、观察格式和答案提交规则等环境因素影响。这些细节常未明确说明,导致结果难以比较或复现。我们提出SimpleWikiSearch,其语料构建、检索栈、工具契约和评估协议均清晰可运行。系统从完整英文维基导出数据开始,清洗并切块语料,建立关键词与密集检索索引,并提供由 exttt{search}、 exttt{open_url}和 exttt{submit_answer}组成的最小工具接口。我们在六个QA数据集上报告了使用开源LLM的基线结果,并提供随机300个样本用于与闭源商业模型对比。SimpleWikiSearch提供领域专用的智能体测试平台和可控离线环境,实现可复现的智能体搜索评估。其贡献在于明确的参考设置,而非新算法。代码与数据将公开于:https://github.com/JimXiongGM/simple_wiki_search。

原文摘要 · Abstract (English)

Large language model (LLM)-based agentic search systems are often evaluated as if the underlying LLM were the only component that matters, yet their measured performance also depends on the surrounding search environment: the Wikipedia snapshot, preprocessing pipeline, chunking policy, retrieval backend, tool schema, observation format, and answer submission rule. These details are frequently under-specified, making it difficult to compare results or reproduce reported baselines. We present SimpleWikiSearch, whose corpus construction, retrieval stack, tool contract, and evaluation protocol are explicit and runnable. The environment starts from a full English Wikipedia dump, cleans and chunks the corpus, builds keyword and dense retrieval indexes, and exposes a minimal tool interface consisting of \texttt{search}, \texttt{open\_url}, and \texttt{submit\_answer}. We report baseline results on six QA datasets using open-source LLMs and provide a random-300 subset for comparisons with closed-source commercial models. SimpleWikiSearch provides a domain-specific agent harness and a controlled offline environment for reproducible agentic-search evaluation. Its contribution is this specified reference setup, rather than a new agent algorithm. Code and data will be available at: https://github.com/JimXiongGM/simple_wiki_search.

智能体搜索维基检索可复现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。