用AI生成葡萄牙语阅读题，质量接近人工，但选项设计仍有缺陷。
From Model to Classroom: Evaluating Generated MCQs for Portuguese with Narrative and Difficulty Concerns
- 基于课程叙事和难度分级，用AI生成葡萄牙语选择题。
- 专家评审与学生答题数据验证，生成题质量与人工题相当。
- 生成选项语义不清、干扰项难吸引学生，适合教育AI研究者参考。
尽管选择题在学习与评估中价值显著，但手动创作符合不同难度等级和阅读能力要求的题目仍耗时费力。近期生成式AI为高效自动化生成选择题提供了可能，但对生成题目实际质量与可靠性的评估仍关注不足，尤其是在生成失败的情况下。这一问题在真实教学场景应用中尤为关键。此外，多数生成研究集中于英语，其他语言研究较少。本文探讨当前生成模型在葡萄牙语阅读理解题生成中的表现，该语言具有丰富的词形变化特征。研究聚焦于生成与课程相关叙事元素一致、涵盖不同难度层级的题目，并通过专家评审及学生答题数据提取的心理测量特性，评估其对小学阶段学生的适用性。结果显示，当前模型生成的题目质量可与人工题媲美；但存在语义清晰度不足与答案可答性问题，且在生成能有效吸引学生的干扰项方面仍面临挑战。
原文摘要 · Abstract (English)
While MCQs are valuable for learning and evaluation, manually creating them with varying difficulty levels and targeted reading skills remains a time-consuming and costly task. Recent advances in generative AI provide an opportunity to automate MCQ generation efficiently. However, assessing the actual quality and reliability of generated MCQs has received limited attention -- particularly regarding cases where generation fails. This aspect becomes particularly important when the generated MCQs are meant to be applied in real-world settings. Additionally, most MCQ generation studies focus on English, leaving other languages underexplored. This paper investigates the capabilities of current generative models in producing MCQs for reading comprehension in Portuguese, a morphologically rich language. Our study focuses on generating MCQs that align with curriculum-relevant narrative elements and span different difficulty levels. We evaluate these MCQs through expert review and by analyzing the psychometric properties extracted from student responses to assess their suitability for elementary school students. Our results show that current models can generate MCQs of comparable quality to human-authored ones. However, we identify issues related to semantic clarity and answerability. Also, challenges remain in generating distractors that engage students and meet established criteria for high-quality MCQ option design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。