研究学生用AI写代码，发现聊天式交互更优且效率更高。
Analysis of Student-LLM Interaction in a Software Engineering Project
- 通过分析126名学生与AI助手的13周互动，对比不同工具使用效果。
- 学生更偏好ChatGPT，其生成代码复杂度更低，质量更高。
- 早期接触AI对培养未来工程师至关重要，适合教育研究者参考。
大型语言模型（LLMs）在多个领域表现日益出色，教育界正积极探索将其融入学习过程。尤其在软件工程中，LLMs在代码总结、生成和调试方面展现出显著优势。然而，现有研究多聚焦于技术应用，缺乏对学生学习过程影响的深入探讨。为此，我们分析了126名本科生在13周课程中与AI助手的互动，涵盖对话记录、生成代码、实际使用代码及人工干预程度。结果表明，学生更倾向使用ChatGPT，其生成代码的计算复杂度低于CoPilot。此外，基于对话的交互方式相比自动生成功能，能显著提升代码质量。研究强调，尽早将LLMs引入软件工程教育，对培养学生竞争力至关重要，未来工程师需掌握与AI协作的能力。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are becoming increasingly competent across various domains, educators are showing a growing interest in integrating these LLMs into the learning process. Especially in software engineering, LLMs have demonstrated qualitatively better capabilities in code summarization, code generation, and debugging. Despite various research on LLMs for software engineering tasks in practice, limited research captures the benefits of LLMs for pedagogical advancements and their impact on the student learning process. To this extent, we analyze 126 undergraduate students' interaction with an AI assistant during a 13-week semester to understand the benefits of AI for software engineering learning. We analyze the conversations, code generated, code utilized, and the human intervention levels to integrate the code into the code base. Our findings suggest that students prefer ChatGPT over CoPilot. Our analysis also finds that ChatGPT generates responses with lower computational complexity compared to CoPilot. Furthermore, conversational-based interaction helps improve the quality of the code generated compared to auto-generated code. Early adoption of LLMs in software engineering is crucial to remain competitive in the rapidly developing landscape. Hence, the next generation of software engineers must acquire the necessary skills to interact with AI to improve productivity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。