研究AI写代码提交的描述风格如何影响人类评审反应
How AI Coding Agents Communicate: A Study of Pull Request Description Characteristics and Human Review Responses
- 对比5个AI编码代理的提交描述结构差异
- 不同代理的描述风格影响评审响应速度与合并率
- 揭示人机协作中提交质量的关键作用
大规模语言模型的快速应用催生了能够自主在GitHub上创建代码提交（Pull Request）的AI编码代理。然而，这些代理在提交描述特征上的差异及其对人类评审的影响仍不明确。本研究基于AIDev数据集，对五个AI编码代理生成的提交进行了实证分析，考察其描述的结构性特征，并评估人类评审者在评审活跃度、响应时间、情感倾向及合并结果方面的反应。结果发现，不同代理的提交描述风格存在显著差异，且这些差异与评审参与度、响应时长和合并成功率密切相关。各代理在评审互动指标和合并率方面表现不一，表明提交描述的质量与呈现方式在人机协同开发中起关键作用。
原文摘要 · Abstract (English)
The rapid adoption of large language models has led to the emergence of AI coding agents that autonomously create pull requests on GitHub. However, how these agents differ in their pull request description characteristics, and how human reviewers respond to them, remains underexplored. In this study, we conduct an empirical analysis of pull requests created by five AI coding agents using the AIDev dataset. We analyze agent differences in pull request description characteristics, including structural features, and examine human reviewer response in terms of review activity, response timing, sentiment, and merge outcomes. We find that AI coding agents exhibit distinct PR description styles, which are associated with differences in reviewer engagement, response time, and merge outcomes. We observe notable variation across agents in both reviewer interaction metrics and merge rates. These findings highlight the role of pull request presentation and reviewer interaction dynamics in human-AI collaborative software development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。