arXiv:2501.10091cs.HCcs.AI2025-01中稿 · ACM International …被引 51

37名编程学生用AI写代码,多数直接要完整答案,反而越错越依赖。

How Do Programming Students Use Generative AI?

  • 通过监控实验观察学生使用ChatGPT的真实行为。
  • 23人使用AI,其中多数直接请求生成完整代码。
  • 学生易陷入错误代码反复求助的恶性循环,缺乏批判性思考。

编程学生广泛使用ChatGPT等强大生成式AI工具。尽管这些工具有助于理解学习内容并辅助练习,但教育者日益担忧学生过度依赖生成结果,导致批判性思维能力下降。为探究学生实际使用方式及其对学习行为的影响,我们开展了一项包含37名编程学生的探索性实验,提供受监控的ChatGPT访问权限,要求完成一项需代码理解与推理的编程任务(非直接可解)。仅23名学生实际使用聊天机器人,多数人请求生成完整解决方案。观察到两种主要使用策略:获取通用概念知识、直接生成代码。学生很少利用AI理解自身代码错误或提升理解力,反而陷入提交错误代码后不断求助修复的恶性循环。自述经常使用生成式AI的学生更倾向于请求完整答案。研究结果表明,对生成式AI可能削弱程序员自主性与效率的担忧确有依据。我们讨论了研究人员与教育者如何应对学生无批判地过度依赖生成式AI的风险,并提出适用于大规模复现的研究设计改进建议。

原文摘要 · Abstract (English)

Programming students have a widespread access to powerful Generative AI tools like ChatGPT. While this can help understand the learning material and assist with exercises, educators are voicing more and more concerns about an overreliance on generated outputs and lack of critical thinking skills. It is thus important to understand how students actually use generative AI and what impact this could have on their learning behavior. To this end, we conducted a study including an exploratory experiment with 37 programming students, giving them monitored access to ChatGPT while solving a code authoring exercise. The task was not directly solvable by ChatGPT and required code comprehension and reasoning. While only 23 of the students actually opted to use the chatbot, the majority of those eventually prompted it to simply generate a full solution. We observed two prevalent usage strategies: to seek knowledge about general concepts and to directly generate solutions. Instead of using the bot to comprehend the code and their own mistakes, students often got trapped in a vicious cycle of submitting wrong generated code and then asking the bot for a fix. Those who self-reported using generative AI regularly were more likely to prompt the bot to generate a solution. Our findings indicate that concerns about potential decrease in programmers' agency and productivity with Generative AI are justified. We discuss how researchers and educators can respond to the potential risk of students uncritically over-relying on Generative AI. We also discuss potential modifications to our study design for large-scale replications.

生成式AI编程教育学习行为

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。