用低成本模型探索代码库,能保留90%以上定位精度
Cost-Effective Repository Exploration for Agentic Issue Localization

- 用低开销模型替代高成本模型进行代码库文件筛选
- 节省84%-95%的调用次数,时间减少41%-88%
- 适合对定位精度要求不高的自动化开发场景
代码库探索是编码代理流程中一个独特且昂贵的阶段:在生成补丁前,代理必须识别哪些文件可能相关。我们研究是否可将此阶段交由低成本模型完成,同时保持有效的定位质量。基于IssueLoc-Bench,在499个SWE-bench Verified衍生任务及153个额外仓库的500个任务上,评估五种探索模型在相同只读交互界面下的表现。衡量早期候选发现、前三名正确文件覆盖率、严格文件集恢复率、代理耗时与令牌使用量,并进行实例级不确定性与仓库聚类敏感性分析。最优探索模型在定位指标上领先,但更低成本方案亦表现出色:根据模型和评估维度,低成本探索者保留约78-94%的参考Hit@3与73-92%的F1,同时平均代理时间减少41%-88%,令牌消耗降低84%-95%。最优操作点取决于下游使用方式:排序与覆盖率反映可移交候选的恢复能力,而F1与精确匹配则刻画严格的文件准入门槛。结果支持将代码库探索视为可独立度量与预算化的模块化编码代理阶段,其选择应依据下游交接协议。
原文摘要 · Abstract (English)
Repository exploration is a distinct and costly stage of coding-agent pipelines: before generating a patch, an agent must identify which repository files are likely to matter. We study whether this stage can be delegated to lower-cost models while retaining useful localization quality. Using our IssueLoc-Bench, we evaluate five explorer models under the same read-only interactive interface on 499 SWE-bench Verified-derived tasks and 500 tasks from 153 additional repositories. We measure early candidate discovery, top-three gold-file coverage, strict file-set recovery, agent time, and token usage, with paired instance-level uncertainty and repository-clustered sensitivity analysis. The highest-quality explorer leads across localization metrics, but substantially cheaper operating points emerge: depending on the model and evaluation arm, lower-cost explorers retain approximately 78-94% of the reference Hit@3 and 73-92% of its F1 while reducing mean agent time by 41-88% and token usage by 84-95%. The preferred operating point depends on how localization is consumed downstream: ranking and coverage metrics characterize recoverable candidate handoffs, whereas F1 and exact match characterize restrictive file gates. These results support treating repository exploration as an independently measurable and budgetable stage of modular coding agents, with explorer selection guided by the downstream handoff contract.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。