分析39项研究，发现大模型助手显著提升开发效率但可能影响代码质量与团队协作。
The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Study
- 系统梳理39篇论文，评估大模型助手对开发效率的影响
- 90%研究显示效率提升，但代码质量评价结果矛盾
- 多数研究缺乏长期和团队层面评估，需更全面的评测框架
大型语言模型助手（LLM-assistants）正重塑软件开发方式。开发者在编码、测试、调试、文档编写和设计等任务中日益依赖此类工具。然而，尽管关注度持续上升，现有研究尚未整合其对开发人员生产力的影响。本文对2014年1月至2024年12月间发表的39篇同行评审研究进行了系统性综述与映射分析。结果显示，多数研究报告了显著收益，如开发速度加快、代码搜索成本降低及重复任务自动化；但也有研究指出认知负荷转移和团队协作减弱的风险。目前关于LLM助手是否提升代码质量仍无定论，因研究结果因上下文和评估标准不同而相互矛盾。尽管90%的研究采用至少两个SPACE维度（满意度、性能、效率等），但仅15%超过三个维度，表明综合评估仍有较大空白。其中，满意度、性能和效率被最常考察，而沟通与活动维度仍被忽视。多数研究为探索性且方法多样，但缺乏纵向与团队级实验。本研究揭示关键研究缺口，并提出未来研究与实践建议。所有相关资料已公开于https://zenodo.org/records/18489222。
原文摘要 · Abstract (English)
Large language model assistants (LLM-assistants) present new opportunities to transform software development. Developers are increasingly adopting these tools across tasks, including coding, testing, debugging, documentation, and design. Yet, despite growing interest, there is no synthesis of how LLM-assistants affect software developer productivity. In this paper, we present a systematic review and mapping of 39 peer-reviewed studies published between January 2014 and December 2024 that examine this impact. Our analysis reveals that the majority of studies report considerable benefits from LLM-assistants, though a notable subset identifies critical risks. Commonly reported gains include accelerated development, minimized code search, and the automation of trivial and repetitive tasks. However, studies also highlight concerns around cognitive offloading and reduced team collaboration. Our study reveals that whether LLM-based assistants improve or degrade code quality remains unresolved, as existing studies report contradictory outcomes contingent on context and evaluation criteria. While the majority of studies (90%) adopt a multi-dimensional perspective by examining at least two SPACE dimensions, reflecting increased awareness of the complexity of developer productivity, only 15% extend beyond three dimensions, indicating substantial room for more integrated evaluations. Satisfaction, Performance, and Efficiency are the most frequently investigated dimensions, whereas Communication and Activity remain underexplored. Most studies are exploratory (59%) and methodologically diverse, but lack longitudinal and team-based evaluations. This review surfaces key research gaps and provides recommendations for future research and practice. All artifacts associated with this study are publicly available at https://zenodo.org/records/18489222
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。