LLM让程序员提效9.6%，新手增产，高手学得更快。
Beyond Code: The Multidimensional Impacts of Large Language Models in Software Development
- 用意大利禁用ChatGPT做自然实验，对比三国开发者数据。
- 使用LLM后，开发效率提升6.4%，知识共享增9.6%，技能提升8.4%。
- 复杂或快速变化的项目中，学习效果最明显，适合团队协作场景。
大型语言模型（LLMs）正深刻影响软件开发，尤其在开源软件（OSS）领域。本文梳理了LLM通过代码生成、知识传递和技能提升三方面影响开源开发的机制，并基于意大利临时禁用ChatGPT的自然实验，采用双重差分法分析意大利、法国、葡萄牙三国共88,022名GitHub开源开发者数据。结果表明，接触ChatGPT使开发者生产力提升6.4%，知识共享增加9.6%，技能习得提高8.4%。该效应随经验水平分化：新手主要获益于生产力提升，资深开发者则更受益于知识共享与技能加速。此外，LLM学习效果高度依赖上下文，在技术复杂、碎片化或快速演进的场景下表现最优。研究揭示，LLM带来的生产率提升不仅来自代码生成，还源于协作学习与知识交换，对理解其在开源生态中的全面影响至关重要。管理启示：合理部署可加速新人上手、促进中间层协作，助力组织长期提质增效。
原文摘要 · Abstract (English)
Large language models (LLMs) are poised to significantly impact software development, especially in the Open-Source Software (OSS) sector. To understand this impact, we first outline the mechanisms through which LLMs may influence OSS through code development, collaborative knowledge transfer, and skill development. We then empirically examine how LLMs affect OSS developers' work in these three key areas. Leveraging a natural experiment from a temporary ChatGPT ban in Italy, we employ a Difference-in-Differences framework with two-way fixed effects to analyze data from all OSS developers on GitHub in three similar countries, Italy, France, and Portugal, totaling 88,022 users. We find that access to ChatGPT increases developer productivity by 6.4%, knowledge sharing by 9.6%, and skill acquisition by 8.4%. These benefits vary significantly by user experience level: novice developers primarily experience productivity gains, whereas more experienced developers benefit more from improved knowledge sharing and accelerated skill acquisition. In addition, we find that LLM-assisted learning is highly context-dependent, with the greatest benefits observed in technically complex, fragmented, or rapidly evolving contexts. We show that the productivity effects of LLMs extend beyond direct code generation to include enhanced collaborative learning and knowledge exchange among developers, dynamics that are essential for gaining a holistic understanding of LLMs' impact in OSS. Our findings offer critical managerial implications: strategically deploying LLMs can accelerate novice developers' onboarding and productivity, empower intermediate developers to foster knowledge sharing and collaboration, and support rapid skill acquisition, together enhancing long-term organizational productivity and agility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。