测试编码代理能否通用,发现其在复杂任务中表现受限。
Can Coding Agents Be General Agents?

- 用真实企业系统中的业务任务评估编码代理
- 简单任务完成可靠,复杂任务常因逻辑与代码衔接失败
- 适合研究通用AI代理的局限性或企业自动化落地者
随着编码代理能力快速提升,用户已将其应用于软件工程以外的通用任务。本文探讨编码代理是否可成功实现端到端的企业流程自动化。我们指出当前评估存在的不足,并在一个开源的企业资源规划(open-core ERP)系统中开展案例研究,评估编码代理在实际业务任务中的表现。结果表明,该代理在简单任务上表现稳定,但在复杂任务中出现典型失败,反映出在领域逻辑与代码执行之间建立有效衔接是通用化的主要瓶颈。
原文摘要 · Abstract (English)
As coding agents have seen rapid capability and adoption gains, users are applying them to general tasks beyond software engineering. In this post, we investigate whether coding agents can successfully generalize to end-to-end business process automation. We identify gaps in current evaluations, and conduct a case study to evaluate a coding agent on practical business tasks in an open-core Enterprise Resource Planning system. We find that the agent reliably completes simple tasks but exhibits characteristic failures on complex tasks, suggesting that bridging domain logic and code execution is a key bottleneck to generalizability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。