arXiv:2602.02138cs.SEcs.AI2026-02被引 1

提出首个因果分析框架,量化多智能体代码生成中各中间输出的重要性。

CAM: A Causality-based Analysis Framework for Multi-Agent Code Generation Systems

  • 基于因果分析,系统评估中间特征对正确性的贡献度。
  • 发现交互依赖特征,混合后端模型可提升7.3%通过率。
  • 可用于故障修复与特征剪枝,提升效率且不降性能。

尽管多智能体代码生成系统(MACGS)已取得显著进展,但其复杂架构产生的大量中间输出对系统正确性的影响尚不明确,阻碍了针对性优化。为此,我们提出首个基于因果分析的MACGS框架CAM,系统量化不同中间特征对系统正确性的贡献。通过全面分类中间输出并模拟真实错误,识别关键特征并聚合重要性排名。实证分析揭示:首先,存在上下文依赖特征——其重要性主要由与其他特征的交互产生,提示质量保障需加入跨特征一致性检查;其次,按能力分配不同后端大模型的混合架构可实现最高7.3%的Pass@1提升,表明该方向具有潜力。进一步验证了CAM的实用性:(1)故障修复通过优化前3个高重要性特征,成功率高达73.6%;(2)特征剪枝可减少最多33.6%的中间标记消耗,同时保持生成性能。本工作为MACGS设计与部署提供可操作洞察,确立因果分析在理解与改进MACGS中的强大作用。

原文摘要 · Abstract (English)

Despite the remarkable success that Multi-Agent Code Generation Systems (MACGS) have achieved, the inherent complexity of multi-agent architectures produces substantial volumes of intermediate outputs. To date, the individual importance of these intermediate outputs to the system correctness remains opaque, which impedes targeted optimization of MACGS designs. To address this challenge, we propose CAM, the first \textbf{C}ausality-based \textbf{A}nalysis framework for \textbf{M}ACGS that systematically quantifies the contribution of different intermediate features for system correctness. By comprehensively categorizing intermediate outputs and systematically simulating realistic errors on intermediate features, we identify the important features for system correctness and aggregate their importance rankings. We conduct extensive empirical analysis on the identified importance rankings. Our analysis reveals intriguing findings: first, we uncover context-dependent features\textemdash features whose importance emerges mainly through interactions with other features, revealing that quality assurance for MACGS should incorporate cross-feature consistency checks; second, we reveal that hybrid backend MACGS with different backend LLMs assigned according to their relative strength achieves up to 7.3\% Pass@1 improvement, underscoring hybrid architectures as a promising direction for future MACGS design. We further demonstrate CAM's practical utility through two applications: (1) failure repair which achieves a 73.6\% success rate by optimizing top-3 importance-ranked features and (2) feature pruning that reduces up to 33.6\% intermediate token consumption while maintaining generation performance. Our work provides actionable insights for MACGS design and deployment, establishing causality analysis as a powerful approach for understanding and improving MACGS.

多智能体代码生成因果分析性能优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。