用多维度推理提升代码智能体的上下文过滤效率
Context Pruning for Coding Agents via Multi-Rubric Latent Reasoning
- 将代码相关性拆分为语义证据与依赖支持两维度,分别建模
- 在4个基准上实现12胜16,单轮任务准确率最高提升3.5%
- 无需额外标注,通过程序分析自动生成多维标签
基于大模型的代码智能体耗费大量令牌读取代码库文件,但其中多数内容与任务无关。现有方法采用单一目标序列标注器压缩上下文,将所有相关性特征合并为单一得分和转移矩阵,造成建模瓶颈:单一CRF转移先验难以适应连续语义段与稀疏结构支撑行等异质保留模式。本文提出LaMR(Latent Multi-Rubric)框架,将代码相关性分解为语义证据与依赖支持两个可解释维度,每个维度由专用的CRF建模,具备特定转移动态。专家混合门控网络根据查询动态加权各维度输出,最终融合后由一阶CRF生成保留或丢弃决策。为避免额外标注成本,利用基于AST的程序分析从训练语料中自动推导多维标签,同时对教师标签进行去噪。实验表明,LaMR在四个基准(SWE-Bench Verified、SWE-QA、LCC、LongCodeQA)上16次对比中有12次胜出,多轮任务最多节省31%令牌,单轮任务精确匹配率最高提升+3.5%,且因上下文去噪性能常优于全量上下文基线,余下性能下降均在边际范围内。
原文摘要 · Abstract (English)
LLM-powered coding agents spend the majority of their token budget reading repository files, yet much of the retrieved code is irrelevant to the task at hand. Existing learned pruners compress this context with a single-objective sequence labeler, collapsing all facets of code relevance into one score and one transition matrix. We show that this formulation creates a modeling bottleneck: a single CRF transition prior must serve heterogeneous retention patterns, including contiguous semantic spans and sparse structural support lines. We propose LaMR (Latent Multi-Rubric), a structured pruning framework that decomposes code relevance into two interpretable quality dimensions, semantic evidence and dependency support, each modeled by a dedicated CRF with dimension-specific transition dynamics. A mixture-of-experts gating network dynamically weights the per-rubric emissions conditioned on the query, and a final CRF layer on the fused emissions produces the aggregate keep-or-prune decision. To supervise each dimension without additional annotation cost, we derive multi-rubric labels from the existing training corpus via AST-based program analysis, simultaneously denoising the teacher's binary labels. By effectively filtering distracting noise, LaMR frequently matches or even outperforms unpruned full-context baselines. Experiments on four benchmarks (SWE-Bench Verified, SWE-QA, LCC, LongCodeQA) show that LaMR wins 12 of 16 head-to-head multi-turn comparisons. It saves up to 31% more tokens on multi-turn agent tasks and improves Exact Match by up to +3.5 on single-turn tasks, while performance is frequently enhanced by denoising the context, and any remaining drops are marginal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。