arXiv:2603.24940cs.PLcs.AI2026-03被引 8

混合AI反馈系统显著提升编程学习效果,优于纯自适应或纯生成式方案。

Evaluating adaptive and generative AI-based feedback and recommendations in a knowledge-graph-integrated programming learning system

  • 融合知识图谱与用户历史的生成式反馈机制
  • 混合模式下正确代码提交率最高,错误尝试最少
  • 生成式反馈广受好评,适合编程教育场景

本文设计并开发了一个将大语言模型(LLM)与检索增强生成(RAG)结合的框架,利用知识图谱和用户交互历史,集成至已有自适应学习支持系统中,用于评估代码、生成形成性反馈并推荐练习题。研究还比较了三种教学模式:自适应、生成式AI(GenAI)和混合式GenAI-自适应模式。基于4956次代码提交的四个关键日志特征,实验分析显示,接受GenAI反馈的学习者在正确代码数量上显著更多,且缺失关键编程逻辑的提交更少。其中,混合模式在正确提交数和错误/不完整尝试数方面均表现最佳,优于仅自适应或仅GenAI模式。问卷调查进一步表明,生成式反馈普遍被认为有帮助,所有模式在易用性和实用性上均获积极评价。结果表明,混合式GenAI-自适应模式在所有测量指标上均优于其他两种模式。

原文摘要 · Abstract (English)

This paper introduces the design and development of a framework that integrates a large language model (LLM) with a retrieval-augmented generation (RAG) approach leveraging both a knowledge graph and user interaction history. The framework is incorporated into a previously developed adaptive learning support system to assess learners' code, generate formative feedback, and recommend exercises. Moerover, this study examines learner preferences across three instructional modes; adaptive, Generative AI (GenAI), and hybrid GenAI-adaptive. An experimental study was conducted to compare the learning performance and perception of the learners, and the effectiveness of these three modes using four key log features derived from 4956 code submissions across all experimental groups. The analysis results show that learners receiving feedback from GenAI modes had significantly more correct code and fewer code submissions missing essential programming logic than those receiving feedback from adaptive mode. In particular, the hybrid GenAI-adaptive mode achieved the highest number of correct submissions and the fewest incorrect or incomplete attempts, outperforming both the adaptive-only and GenAI-only modes. Questionnaire responses further indicated that GenAI-generated feedback was widely perceived as helpful, while all modes were rated positively for ease of use and usefulness. These results suggest that the hybrid GenAI-adaptive mode outperforms the other two modes across all measured log features.

编程教育生成式AI知识图谱自适应学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。