让人类与AI在软件团队中平等协作,实测效果媲美甚至超越传统AI系统。
ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teams
- 设计可混合作业的协作框架,支持人与AI在Slack中自主分工、沟通与执行。
- 在游戏开发任务中,AI agent生成代码质量与现有系统相当或更优。
- 提出自动化分析方法,量化不同角色AI的协作行为差异,如CEO发言频率更高。
我们探索了人类与人工智能(AI)之间以团队形式协同工作的潜力,提出并测试了一个通用框架,使人类和多个AI代理能够作为平等成员共同协作。ChatCollab的新颖架构允许代理(无论人类或AI)以任意角色加入协作,自主在Slack中执行任务与沟通,并对合作者是人还是AI保持无感。以软件工程为案例研究,我们发现其AI代理能准确识别自身角色与职责,与其他代理协调工作,并在等待输入或交付物后再继续。相较于三个先前用于软件开发的多智能体系统,ChatCollab的AI代理在互动式游戏开发任务中产出的软件质量相当或更优。我们还提出一种自动化方法分析协作动态,有效识别出具有不同角色特征的代理行为,从而实现实验条件下协作模式的定量比较。例如,在对比中发现,AI CEO代理的建议频率通常比AI产品经理或开发者高出2-4倍,表明ChatCollab中的代理可有意义地承担差异化协作角色。代码与数据见:https://github.com/ChatCollab。
原文摘要 · Abstract (English)
We explore the potential for productive team-based collaboration between humans and Artificial Intelligence (AI) by presenting and conducting initial tests with a general framework that enables multiple human and AI agents to work together as peers. ChatCollab's novel architecture allows agents - human or AI - to join collaborations in any role, autonomously engage in tasks and communication within Slack, and remain agnostic to whether their collaborators are human or AI. Using software engineering as a case study, we find that our AI agents successfully identify their roles and responsibilities, coordinate with other agents, and await requested inputs or deliverables before proceeding. In relation to three prior multi-agent AI systems for software development, we find ChatCollab AI agents produce comparable or better software in an interactive game development task. We also propose an automated method for analyzing collaboration dynamics that effectively identifies behavioral characteristics of agents with distinct roles, allowing us to quantitatively compare collaboration dynamics in a range of experimental conditions. For example, in comparing ChatCollab AI agents, we find that an AI CEO agent generally provides suggestions 2-4 times more often than an AI product manager or AI developer, suggesting agents within ChatCollab can meaningfully adopt differentiated collaborative roles. Our code and data can be found at: https://github.com/ChatCollab.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。