对比了grep与向量检索在智能体搜索中的表现，发现grep更准但结果依赖工具调用方式。
Is Grep All You Need? How Agent Harnesses Reshape Agentic Search
- 用grep和向量检索对比智能体搜索效果，分直接输出与文件读取两种方式
- 在LongMemEval测试中，grep准确率普遍高于向量检索，最高差值达12.3个百分点
- 工具调用方式和框架选择对性能影响显著，甚至超过检索策略本身
大型语言模型（LLM）智能体的发展使得模型能自主完成信息检索、调用工具并推理，以代表用户完成任务。尽管检索增强生成（RAG）在智能体搜索系统中广泛应用，现有研究仍缺乏对检索策略选择如何与智能体架构及工具调用范式相互作用的系统性比较。重要实际维度，如工具输出呈现方式以及在存在更多无关上下文时的搜索表现，尚未得到充分探索。本文开展两项实证研究：实验1在LongMemEval的116个问题样本上，使用自研代理框架Chronos及Claude Code、Codex、Gemini CLI等原生命令行框架，对比grep与向量检索在内联工具结果和分文件读取结果两种场景下的表现；实验2则在逐步增加无关对话历史的情况下，比较仅用grep与仅用向量检索的效果。结果显示，在Chronos与各厂商CLI框架下，grep整体准确率均优于向量检索；然而，最终得分仍高度依赖所用框架与工具调用风格，即使对话数据相同亦如此。
原文摘要 · Abstract (English)
Recent advances in Large Language Model (LLM) agents have enabled complex agentic workflows where models autonomously retrieve information, call tools, and reason over large corpora to complete tasks on behalf of users. Despite the growing adoption of retrieval-augmented generation (RAG) in agentic search systems, existing literature lacks a systematic comparison of how retrieval strategy choice interacts with agent architecture and tool-calling paradigm. Important practical dimensions, including how tool outputs are presented to the model and how performance changes when searches must cope with more irrelevant surrounding text, remain under-explored in agent loops. This paper reports an empirical study organized into two experiments. Experiment 1 compares grep and vector retrieval on a 116-question sample from LongMemEval, using a custom agent harness (Chronos) and provider-native CLI harnesses (Claude Code, Codex, and Gemini CLI), for both inline tool results and file-based tool results that the model reads separately. Experiment 2 compares grep-only and vector-only retrieval while progressively mixing in additional unrelated conversation history, so that each query is embedded in more distracting material alongside the passages that matter. Across Chronos and the provider CLIs, grep generally yields higher accuracy than vector retrieval in our comparisons in experiment 1; at the same time, overall scores still depend strongly on which harness and tool-calling style is used, even when the underlying conversation data are the same.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。