首个大规模实证数据集揭示AI程序员如何改变软件工程
The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering
- 构建包含45万+代码提交的AIDev数据集，记录五大主流AI编码助手真实协作行为
- AI提效显著但接受率低：三天提交量超三年，代码结构更简单
- 适合研究人机协同、AI治理与智能开发工具优化的学者和开发者
软件工程3.0时代正因AI队友崛起而展开：自主、目标驱动的系统正与人类开发者协同工作。其中，自主编码代理尤为关键，现已能大规模主动发起、审查和演进代码。本文提出AIDev，首个大规模真实世界数据集，涵盖五款领先代理（OpenAI Codex、Devin、GitHub Copilot、Cursor、Claude Code）在61,000个仓库、47,000名开发者中产生的456,000+个拉取请求。该数据集提供丰富的元信息，包括提交时间、作者、评审周期、代码变更及集成结果，为基准测试、代理就绪度评估、协作建模与AI治理研究提供实证基础。不同于以往依赖合成基准（如SWE-bench）的研究，AIDev揭示：尽管代理在速度上远超人类，其代码合并率却较低，存在信任与实用性差距；同时，一名开发者三日内提交的代码量相当于过去三年总量，但代码结构更简单（基于代码复杂度指标）。我们期望AIDev成为持续演进的公共资源，支撑下一代人机共生开发范式的研究。数据集已公开于 https://github.com/SAILResearch/AI_Teammates_in_SE3。
原文摘要 · Abstract (English)
The future of software engineering--SE 3.0--is unfolding with the rise of AI teammates: autonomous, goal-driven systems collaborating with human developers. Among these, autonomous coding agents are especially transformative, now actively initiating, reviewing, and evolving code at scale. This paper introduces AIDev, the first large-scale dataset capturing how such agents operate in the wild. Spanning over 456,000 pull requests by five leading agents--OpenAI Codex, Devin, GitHub Copilot, Cursor, and Claude Code--across 61,000 repositories and 47,000 developers, AIDev provides an unprecedented empirical foundation for studying autonomous teammates in software development. Unlike prior work that has largely theorized the rise of AI-native software engineering, AIDev offers structured, open data to support research in benchmarking, agent readiness, optimization, collaboration modeling, and AI governance. The dataset includes rich metadata on PRs, authorship, review timelines, code changes, and integration outcomes--enabling exploration beyond synthetic benchmarks like SWE-bench. For instance, although agents often outperform humans in speed, their PRs are accepted less frequently, revealing a trust and utility gap. Furthermore, while agents accelerate code submission--one developer submitted as many PRs in three days as they had in three years--these are structurally simpler (via code complexity metrics). We envision AIDev as a living resource: extensible, analyzable, and ready for the SE and AI communities. Grounding SE 3.0 in real-world evidence, AIDev enables a new generation of research into AI-native workflows and supports building the next wave of symbiotic human-AI collaboration. The dataset is publicly available at https://github.com/SAILResearch/AI_Teammates_in_SE3. > AI Agent, Agentic AI, Coding Agent, Agentic Coding, Software Engineering Agent
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。