对比代码代理使用语义检索与词法检索的耗 token 情况，发现前者未必更省
Does a Language Server Save Tokens for Coding Agents? A Measurement Methodology and Preliminary Study
- 设计五臂消融实验，量化语义检索与词法检索在完成任务时的 token 消耗差异
- 实测显示语义检索在多数任务中反而多耗 token，仅弱模型在特定任务中节省
- 建议根据任务类型和模型能力动态选择检索方式，而非统一用语义检索
代码代理大多将上下文预算用于检索。词法检索（grep）通用、即时、零配置，但噪声大，无法区分定义、调用与注释。通过语言服务器协议（LSP）的语义检索精准且带类型，但需运行并索引服务器，每个符号有往返开销。尽管普遍宣称语义检索更省 token，却几乎无公开测量验证其在同等任务成功率下的实际差异。本文提出一个度量指标（tokens-to-success），设计五臂消融实验以隔离语义检索的影响，映射三种预设失败模式为可测变量，并报告初步研究（使用 Python 与 TypeScript 仓库；Claude Opus 4.8、Sonnet 4.6、Haiku 4.5）。结果表明：语义检索在符号定位任务中反而增加消耗（+6% 至 +118%），代理在免费时忽略其使用；在引用完整性上提升精度但未节省 token，也无法突破由代理细致程度决定的召回上限；仅对最弱模型有节流效果。工具选择依赖任务：模型默认在定位任务中使用 grep（0-6% 使用语义），但在引用任务中半数情况下自发调用 LSP。在真实测试执行的编辑任务中差距最显著：grep 完美处理多文件重命名，仅定位的 LSP 在 3/4 情况下因遗漏调用点失败；即使完整、索引预热、文本增强的 LSP（每条引用行内嵌）也难以完全弥补，因重命名需修改注释与字符串，而语义引用不覆盖这些内容。结论并非始终用 LSP，而是应基于任务类别、模型能力与词法噪声，建立自适应路由机制。
原文摘要 · Abstract (English)
Coding agents spend most of their context budget on retrieval. Lexical retrieval (grep) is universal, instant, and zero-setup, but noisy: it cannot tell a definition from a call from a comment. Semantic retrieval via the Language Server Protocol (LSP) is precise and typed, but needs a running, indexed server and pays a per-symbol round-trip. The claim that semantic retrieval is more token-efficient is, we find, asserted almost everywhere and measured almost nowhere: no public source isolates the LSP-vs-lexical token delta for an agent at equal task-success. This paper formalizes the question with one metric (tokens-to-success), specifies a five-arm ablation isolating semantic retrieval from confounds, maps three pre-stated failure modes onto measurable variables, and reports a preliminary study (Python and TypeScript repos; Claude Opus 4.8, Sonnet 4.6, Haiku 4.5). The answer is conditional and usually negative. On symbol-named localization the LSP costs tokens (+6% to +118%) and the agent ignores it when free. On reference-completeness it buys precision but not token savings and cannot raise the recall ceiling set by agent thoroughness; it saves tokens only for the weakest model. Tool choice is task-dependent: models default to grep on localization (0-6% semantic use) but reach for the LSP about half the time on reference tasks, unprompted. On edits scored by real test execution the gap is starkest: grep solves multi-file renames perfectly, a location-only LSP fails three-quarters of them by missing a call site, and even a complete, index-warmed, text-enriched LSP (each reference's line inline, as production LSP-MCP servers do) recovers most of the gap but cannot close it, since a rename must touch comments and strings that semantic references exclude. The implication is not LSP-always but an adaptive router keyed on task class, model capability, and lexical noise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。