用高价模型指挥低价模型完成编程任务，发现有效但受限于训练方式。
Can AI Models Direct Each Other? Organizational Structure as a Probe into Training Limitations
- 高价模型只负责分析与规划，低价模型执行代码，分工降低开销。
- 强管理弱执行时成功率62%，接近单个强模型的60%。
- 指挥需真实能力差，否则反而拖累效率，适合研究多智能体协作机制者。
能否让昂贵的AI模型有效指导廉价模型完成软件工程任务？我们通过引入ManagerWorker双智能体流程进行研究：昂贵的‘管理者’模型（仅文本，无代码执行）分析问题、分配探索任务并审查实现；廉价的‘工作者’模型（具备完整仓库访问权限）执行代码变更。在SWE-bench Lite的200个实例上，评估了五种配置，涵盖管理者-工作者关系、流水线复杂度及模型组合差异。结果表明：（1）强管理者指导弱工作者（62%）成功率接近单一强模型（60%），证明高成本推理可替代高成本执行；（2）弱管理者指导弱工作者（42%）表现低于单独使用弱模型（44%），说明指挥关系需真实能力差距，否则仅为冗余开销；（3）管理者价值在于主动引导而非简单审查——仅审查循环仅提升2个百分点，而结构化探索与规划提升11个百分点，表明主动指挥才是关键；（4）根本原因在于当前模型训练为单一整体代理，拆分为指挥/执行角色违背其训练分布。该流程成功源于设计绕过此矛盾：保持管理者专注文本生成、工作者专注工具使用，并将组织结构外化至代码。这一诊断揭示了训练缺失：委托、范围执行和模式切换等能力尚未纳入训练数据。
原文摘要 · Abstract (English)
Can an expensive AI model effectively direct a cheap one to solve software engineering tasks? We study this question by introducing ManagerWorker, a two-agent pipeline where an expensive "manager" model (text-only, no code execution) analyzes issues, dispatches exploration tasks, and reviews implementations, while a cheap "worker" model (with full repo access) executes code changes. We evaluate on 200 instances from SWE-bench Lite across five configurations that vary the manager-worker relationship, pipeline complexity, and model pairing. Our findings reveal both the promise and the limits of multi-agent direction: (1) a strong manager directing a weak worker (62%) matches a strong single agent (60%) at a fraction of the strong-model token usage, showing that expensive reasoning can substitute for expensive execution; (2) a weak manager directing a weak worker (42%) performs worse than the weak agent alone (44%), demonstrating that the directing relationship requires a genuine capability gap--structure without substance is pure overhead; (3) the manager's value lies in directing, not merely reviewing--a minimal review-only loop adds just 2pp over the baseline, while structured exploration and planning add 11pp, showing that active direction is what makes the capability gap productive; and (4) these behaviors trace to a single root cause: current models are trained as monolithic agents, and splitting them into director/worker roles fights their training distribution. The pipeline succeeds by designing around this mismatch--keeping each model close to its trained mode (text generation for the manager, tool use for the worker) and externalizing organizational structure to code. This diagnosis points to concrete training gaps: delegation, scoped execution, and mode switching are skills absent from current training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。