分析AI生成题解的可读性与难度适应性，提升教育反馈质量。
Analyzing Feedback Mechanisms in AI-Generated MCQs: Insights into Readability, Lexical Properties, and Levels of Challenge
- 用Gemini模型生成三类语气的题目反馈，分难易度分析语言特征。
- 反馈可读性误差仅2.0，词汇丰富度预测误差0.03，表现精准。
- 揭示语气与难度的互动效应，适合教育AI研发者参考。
人工智能在教育场景中生成反馈已引起广泛关注，因其可能提升学习效果。然而，对AI生成反馈的语言特征——包括可读性、词汇丰富度及不同难度下的适应性——仍缺乏全面理解。本研究分析了Google Gemini 1.5-flash文本模型为计算机科学多选题生成的反馈，基于超过1,200道题目，涵盖易、中、难三个难度等级和支持性、中性、挑战性三种语气。计算并评估了长度、可读性（Flesch-Kincaid Grade Level）、词汇丰富度与词汇密度等关键语言指标。采用微调的RoBERTa多任务学习模型预测这些属性，可读性预测的平均绝对误差（MAE）为2.0，词汇丰富度为0.03。结果表明，反馈语气与题目难度存在显著交互作用，显示AI反馈在多样化教学情境中的动态适配能力。研究为构建更个性化、高效的AI驱动反馈机制提供依据，强调其在提升学习效果的同时，也需重视设计与部署中的伦理问题。
原文摘要 · Abstract (English)
Artificial Intelligence (AI)-generated feedback in educational settings has garnered considerable attention due to its potential to enhance learning outcomes. However, a comprehensive understanding of the linguistic characteristics of AI-generated feedback, including readability, lexical richness, and adaptability across varying challenge levels, remains limited. This study delves into the linguistic and structural attributes of feedback generated by Google's Gemini 1.5-flash text model for computer science multiple-choice questions (MCQs). A dataset of over 1,200 MCQs was analyzed, considering three difficulty levels (easy, medium, hard) and three feedback tones (supportive, neutral, challenging). Key linguistic metrics, such as length, readability scores (Flesch-Kincaid Grade Level), vocabulary richness, and lexical density, were computed and examined. A fine-tuned RoBERTa-based multi-task learning (MTL) model was trained to predict these linguistic properties, achieving a Mean Absolute Error (MAE) of 2.0 for readability and 0.03 for vocabulary richness. The findings reveal significant interaction effects between feedback tone and question difficulty, demonstrating the dynamic adaptation of AI-generated feedback within diverse educational contexts. These insights contribute to the development of more personalized and effective AI-driven feedback mechanisms, highlighting the potential for improved learning outcomes while underscoring the importance of ethical considerations in their design and deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。