用强弱模型协作生成代码，成本降40%性能不降。
An Empirical Study on Strong-Weak Model Collaboration for Repo-level Code Generation
- 弱模型处理简单任务，强模型专注难题，按需分配算力。
- 最优策略使成本降低40%，性能接近纯强模型。
- 适合预算有限但需高效代码生成的开发者团队。
我们研究了在仓库级代码生成中，强弱语言模型协同的成本效益问题。弱模型负责低复杂度任务以降低开销，而最困难的任务则交由强模型处理。尽管已有诸多协同架构提出，但很少有工作分析其性能与成本的关系。我们在GitHub问题修复任务上评估了多种协同策略：基于上下文、流水线式和动态调度。最有效的策略在保持与强模型相当性能的同时，将成本降低了40%。基于研究结果，我们提出了在不同预算和性能约束下选择协同策略的实用建议。结果显示，强弱协同能显著提升弱模型表现，仅需少量成本；其中流水线和基于上下文的方法最为高效。代码已开源：https://github.com/shubhamrgandhi/codegen-strong-weak-collab。
原文摘要 · Abstract (English)
We study cost-efficient collaboration between strong and weak language models for repository-level code generation, where the weak model handles simpler tasks at lower cost, and the most challenging tasks are delegated to the strong model. While many works propose architectures for this task, few analyze performance relative to cost. We evaluate a broad spectrum of collaboration strategies: context-based, pipeline-based, and dynamic, on GitHub issue resolution. Our most effective collaborative strategy achieves equivalent performance to the strong model while reducing the cost by 40%. Based on our findings, we offer actionable guidelines for choosing collaboration strategies under varying budget and performance constraints. Our results show that strong-weak collaboration substantially boosts the weak model's performance at a fraction of the cost, pipeline and context-based methods being most efficient. We release the code for our work at https://github.com/shubhamrgandhi/codegen-strong-weak-collab.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。