AI协作提升广告产出效率与文本质量，但影响团队多样性。
Collaborating with AI Agents: Field Experiments on Teamwork, Productivity, and Performance
- 人类与AI组队更聚焦任务，减少人际交流，提高分工效率。
- 每名成员多产50%广告，文本质量更高，图像质量低于人-人组。
- 适合研究人机协作机制或提升内容生产效率的团队参考。
我们通过在自研平台Pairit上开展大规模实验，研究人与AI协作的生产力与绩效机制。将2,234名参与者随机分配至人-人和人-AI团队，共生成11,024条智库广告。通过独立人工评分及在X平台开展的实地实验（约500万次曝光）评估效果。结果显示，人-AI团队每人产出广告量高出50%，文本质量更高，而人-人团队图像质量更优，表明AI能力存在不均衡边界。人-AI团队输出更具同质性。实地实验发现，文本质量提升可增加点击率和观看时长，图像质量提升则降低每次点击成本。三种机制解释了上述结果：一是人-AI协作更任务导向，任务相关消息多25%，人际交流少18%；二是协作中更多委托工作给AI，对AI的直接文本编辑减少62%；三是识别出合作者为AI后，参与者更专注任务并更愿委托工作。任务导向沟通提升广告质量（尤其配合AI），人际沟通则降低质量；委托工作提升文本质量但不影响图像质量，且导致多样性坍缩，形成平均质量更高的同质输出。结果表明，AI通过重塑协作方式，驱动生产效率、表现与输出多样性的变化。
原文摘要 · Abstract (English)
We examined the mechanisms underlying productivity and performance gains from AI agents using a large-scale experiment on Pairit, a platform we developed to study human-AI collaboration. We randomly assigned 2,234 participants to human-human and human-AI teams that produced 11,024 ads for a think tank. We evaluated the ads using independent human ratings and a field experiment on X which garnered ~5M impressions. We found human-AI teams produced 50% more ads per worker and higher text quality, while human-human teams produced higher image quality, suggesting a jagged frontier of AI agent capability. Human-AI teams also produced more homogeneous, or self-similar, outputs. The field experiment revealed higher text quality improved click-through rates and view-through duration, while higher image quality improved cost-per-click rates. We found three mechanisms explained these effects. First, human-AI collaboration was more task-oriented, with 25% more task-oriented messages and 18% fewer interpersonal messages. Second, human-AI collaboration displayed more delegation, as participants delegated 17% more work to AI agents than to human partners and performed 62% fewer direct text edits when working with AI. Third, recognition that the collaborator was an AI moderated these effects as participants who correctly identified they were working with AI were more task-oriented and more likely to delegate work. These mechanisms then explained performance as task-oriented communication improved ad quality, specifically when working with AI, while interpersonal communication reduced ad quality; delegation improved text quality but had no effect on image quality and was positively associated with diversity collapse, creating homogeneous outputs of higher average quality. The results suggest AI agents drive changes in productivity, performance, and output diversity by reshaping teamwork.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。