代码生成模型能力差异大，用结构化架构规范能有效缩小差距。
Architecture as Capability Equalizer for Coding Agents
- 用五种等信息量的架构描述格式对比模型表现，发现格式影响随模型强弱变化。
- 弱模型用结构化格式后，代码质量提升最高达2.42分，最强模型则不受格式影响。
- 适合资源有限的部署场景，尤其对低阶模型提升效果显著。
基于大模型的代码生成代理能从高层描述生成完整软件系统，但架构规范的表达形式如何影响代码质量，以及这种影响是否依赖模型能力，仍不清楚。本文通过控制实验，在六款来自三家厂商（Anthropic Claude、OpenAI GPT、Google Gemini）的模型上，对比了五种信息等价的规范格式（非正式叙述、带约束的Mermaid图与ADR、OpenAPI、C4/Structurizr DSL、带ArchUnit风格规则的TypeScript接口契约）。在90次多轮代理测试中，规范格式与模型能力存在显著交互作用：在最强模型（Sonnet 4.6、GPT-5）上，格式影响极小（质量波动0.17–0.92分）；而在弱模型上，格式差异导致0.83–2.42分的质量差距。使用贴近代码的格式（如OpenAPI、TypeScript契约）可最大程度弥补能力差距。中等能力模型若过度消耗token进入编译调试循环，反而产出更差结果，而强模型能避免此类陷阱。自验证成功率从最强模型的100%（Sonnet）降至最弱模型的0%（Gemini Flash）。对于最弱模型，使用TypeScript契约使API路由覆盖率从33%提升至100%。结构化架构规范具有能力均衡效应，其价值随模型强度降低而上升，对成本优化部署最为有利。
原文摘要 · Abstract (English)
LLM-based coding agents generate complete software systems from high-level descriptions, yet little is known about how the format of architecture specifications affects the quality of generated code or whether this effect depends on model capability. We present a controlled experiment comparing five informationally equivalent specification formats (informal prose, Mermaid diagrams with constraints and ADRs, OpenAPI, C4/Structurizr DSL, and TypeScript interface contracts with ArchUnit-style rules) across six models from three vendor families (Anthropic Claude, OpenAI GPT, Google Gemini). Across 90 multi-turn agent trials, specification format shows a strong format x model interaction. On the strongest models (Sonnet 4.6, GPT-5), format barely matters (quality spread 0.17-0.92). On weaker models, format produces spreads of 0.83-2.42 points, with code-proximate formats (OpenAPI, TypeScript contracts) recovering most of the capability gap. Mid-tier models can consume more tokens than frontier models for worse output when they enter compilation debugging loops that stronger models avoid. Self-validation rates collapse from 100% (Sonnet) to 0% (Gemini Flash) across the capability spectrum. TypeScript contracts triple API route coverage for the weakest model (33% to 100%). Structured architecture specifications serve as a capability equalizer, with value inversely proportional to model strength and the largest returns for cost-optimized deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。