arXiv:2608.04336cs.SEcs.AI2026-08

根据任务难度动态优化代码生成的模型、提示和解码配置,提升效果并降本。

COMPAS: Difficulty-Aware Joint Search for Optimizing Code Generation

  • 按任务难度分组,联合优化提示与解码参数,匹配不同模型表现
  • 在LiveCodeBench上将通过率从45.9%提至52.8%,成本由$36.57降至$4.92
  • 适合追求高效高质代码生成的开发者或系统集成者

代码生成系统每次调用大模型时需选择模型、提示和解码设置。现有优化方法通常仅调整其中部分变量,或对所有任务使用同一配置:全局优化器为所有任务搜索单一配置,路由器仅选择模型,提示优化器则固定模型与解码设置。这使得三者之间的联合、分组交互关系不明确。我们研究发现,提示与解码设置存在交互效应,调参效果随模型变化,最优配置依赖于任务难度。基于此,提出COMPAS(Code-generation Optimization over Models, Prompts, And Decoding Settings),一种难度感知的方法,通过低成本模型筛选和联合提示-解码搜索学习各组的任务质量-成本曲线,并在线路由任务至对应曲线,无需额外搜索。在LiveCodeBench上,相同搜索预算下,COMPAS将pass@1从最佳基线的45.9%提升至52.8%,成本从$36.57降至$4.92。该方法在SWE-bench仓库级生成任务中也表现优异,解决率由70.0%提升至76.0%。代码与可复现资源见https://github.com/gjz78910/COMPAS。

原文摘要 · Abstract (English)

Code generation systems make each LLM call with a model, a prompt, and decoding settings. However, existing optimization methods usually tune only part of these choices or use one fixed configuration for all tasks: global optimizers search one configuration for all tasks, routers choose only a model, and prompt optimizers keep the model and decoding settings fixed. This leaves their joint, group-specific interactions unclear. We therefore examine how these choices interact and observe that prompts and decoding settings interact, tuning effects vary by model, and the best configuration varies by task difficulty. Guided by these observations, we introduce COMPAS (Code-generation Optimization over Models, Prompts, And Decoding Settings), a difficulty-aware method that learns group-specific quality-cost fronts through low-cost model selection and joint prompt-decoding search, then routes each test task to its matching front online without further search. Under a matched search budget on LiveCodeBench, COMPAS improves pass@1 from 45.9% for the best baseline to 52.8% while reducing cost from $36.57 to $4.92. This also transfers to repository-level code generation on SWE-bench, resolving 76.0% of tasks versus 70.0% for the best baseline. Code and the reproducibility artifact are available at https://github.com/gjz78910/COMPAS.

代码生成模型优化提示工程成本效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。