用AI助教提升影视工程硕士教学，实测不影响考试公平性
An AI Teaching Assistant for Motion Picture Engineering
- 基于RAG构建AI助教，针对课程需求优化检索与生成流程
- 43名学生7周内完成296次会话共1889次提问，效果稳定可靠
- 支持开卷考试且无成绩差异，适合想试水AI教学的高校教师
近年来大型语言模型的快速发展推动了以LLM为核心的AI导师实验。然而其具体实现方式及在教学环境中的实际效益仍处于探索初期。本文介绍了在都柏林三一学院影视工程硕士（MPE）课程中，基于检索增强生成（RAG）技术实现的AI教学助教（AI-TA）的具体实施细节，包括提示设计与代码实现，并阐述了为满足课程需求而定制和调优的RAG管道。通过调查工具报告了该系统的影响，实验规模达43名学生、296次会话、1889次查询，持续7周，数据充分可信。与以往研究不同，本实验首次允许学生在开卷考试中使用AI-TA。三次考试的统计分析显示，有无使用AI-TA的学生表现无显著差异（p > 0.05），证明精心设计的评估可保持学术有效性。学生反馈表明，该助教总体有益（均分4.22/5），但对替代人类辅导的态度则褒贬不一（均分2.78/5）。
原文摘要 · Abstract (English)
The rapid rise of LLMs over the last few years has promoted growing experimentation with LLM-driven AI tutors. However, the details of implementation, as well as the benefit in a teaching environment, are still in the early days of exploration. This article addresses these issues in the context of implementation of an AI Teaching Assistant (AI-TA) using Retrieval Augmented Generation (RAG) for Trinity College Dublin's Master's Motion Picture Engineering (MPE) course. We provide details of our implementation (including the prompt to the LLM, and code), and highlight how we designed and tuned our RAG pipeline to meet course needs. We describe our survey instrument and report on the impact of the AI-TA through a number of quantitative metrics. The scale of our experiment (43 students, 296 sessions, 1,889 queries over 7 weeks) was sufficient to have confidence in our findings. Unlike previous studies, we experimented with allowing the use of the AI-TA in open-book examinations. Statistical analysis across three exams showed no performance differences regardless of AI-TA access (p > 0.05), demonstrating that thoughtfully designed assessments can maintain academic validity. Student feedback revealed that the AI-TA was beneficial (mean = 4.22/5), while students had mixed feelings about preferring it over human tutoring (mean = 2.78/5).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。