多代码智能体协作时,缺乏详细规范会导致严重协调失败。
The Specification Gap: Coordination Failure Under Partial Knowledge in Code Agents
- 通过逐步简化接口说明,测试智能体间代码一致性
- 规范越简略,协作准确率从58%降至25%,差距达25--39个百分点
- 清晰规范是解决协调难题的关键,冲突检测无额外帮助
当多个基于大模型的代码智能体独立实现同一类时,必须在规范未明确的内部表示上达成一致。我们研究了51个类生成任务中的协调问题,逐步从完整文档字符串(L0)简化到仅签名(L3),并引入对立结构偏见(列表与字典)以检验集成能力。结果发现:第一,存在持续的规范缺口——双智能体协作准确率从58%降至25%,而单智能体基线仅从89%降至56%,形成25--39个百分点的协调差距,且在两个Claude模型(Sonnet、Haiku)和三次独立运行中保持一致;第二,在最弱规范下,基于AST的冲突检测器达到97%精确度,无需额外大模型调用,但因子恢复实验表明,仅恢复完整规范即可恢复单智能体上限(89%),提供冲突报告无显著增益;第三,将缺口分解为协调成本(+16个百分点)与信息不对称(+11个百分点),二者独立且近似叠加。该结果支持“规范优先”的多智能体代码生成观:更丰富的规范既是主要协调机制,也是充分恢复工具。
原文摘要 · Abstract (English)
When multiple LLM-based code agents independently implement parts of the same class, they must agree on shared internal representations, even when the specification leaves those choices implicit. We study this coordination problem across 51 class-generation tasks, progressively stripping specification detail from full docstrings (L0) to bare signatures (L3), and introducing opposing structural biases (lists vs. dictionaries) to stress-test integration. Three findings emerge. First, a persistent specification gap: two-agent integration accuracy drops from 58% to 25% as detail is removed, while a single-agent baseline degrades more gracefully (89% to 56%), leaving a 25--39 pp coordination gap that is consistent across two Claude models (Sonnet, Haiku) and three independent runs. Second, an AST-based conflict detector achieves 97% precision at the weakest specification level without additional LLM calls, yet a factorial recovery experiment shows that restoring the full specification alone recovers the single-agent ceiling (89%), while providing conflict reports adds no measurable benefit. Third, decomposing the gap into coordination cost (+16 pp) and information asymmetry (+11 pp) suggests that the two effects are independent and approximately additive. The gap is not merely a consequence of hidden information, but reflects the difficulty of producing compatible code without shared decisions. These results support a specification-first view of multi-agent code generation: richer specifications are both the primary coordination mechanism and the sufficient recovery instrument.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。