实测显示AI助写工具让开发提效三成，代码交付量提升28%。
Intuition to Evidence: Measuring AI's True Impact on Developer Productivity
- 在真实企业环境中部署AI平台，整合生成与审查功能。
- 代码评审周期缩短31.8%，高活跃者代码提交量增61%。
- 适合关注落地效果的工程团队和决策者参考。
我们对在企业级规模部署的AI辅助软件开发工具进行了全面的现实世界评估。一年内，多个团队共300名工程师将内部AI平台DeputyDev集成到日常工作中，该平台结合了代码生成与自动化审查功能。通过严谨的队列分析，研究显示生产力显著提升：代码评审周期平均缩短31.8%。开发者采用率高，代码审查功能满意度达85%，93%表示愿继续使用。使用模式呈现系统性增长，从首月4%活跃度上升至第6个月峰值83%，后稳定在60%。顶尖使用者代码推送量提升61%，约30%-40%的生产环境代码由该工具生成，整体代码交付量提升28%。与受控基准测试不同，本研究基于长期纵向分析，提供来自生产环境的真实证据，揭示了将AI融入企业开发流程的变革潜力与实际挑战。
原文摘要 · Abstract (English)
We present a comprehensive real-world evaluation of AI-assisted software development tools deployed at enterprise scale. Over one year, 300 engineers across multiple teams integrated an in-house AI platform (DeputyDev) that combines code generation and automated review capabilities into their daily workflows. Through rigorous cohort analysis, our study demonstrates statistically significant productivity improvements, including an overall 31.8% reduction in PR review cycle time. Developer adoption was strong, with 85% satisfaction for code review features and 93% expressing a desire to continue using the platform. Adoption patterns showed systematic scaling from 4% engagement in month 1 to 83% peak usage by month 6, stabilizing at 60% active engagement. Top adopters achieved a 61% increase in code volume pushed to production, contributing to approximately 30 to 40% of code shipped to production through this tool, accounting for an overall 28% increase in code shipment volume. Unlike controlled benchmark evaluations, our longitudinal analysis provides empirical evidence from production environments, revealing both the transformative potential and practical deployment challenges of integrating AI into enterprise software development workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。