用大模型生成试题，让老师省时省力。
Multiple-Choice Question Generation Using Large Language Models: Methodology and Educator Insights
- 给提示词注入知识，避免大模型胡编乱造。
- 21位教师测试显示GPT-3.5生成题目最有效。
- 适合教育工作者探索AI辅助出题的新方式。
将人工智能融入教育场景正催生新型学习模式，重塑师生实践。其中，大语言模型（LLMs）在生成教学材料和答题支持方面展现出强大潜力，但仍有拓展空间。教师常使用多选题（MCQs）评估学生知识掌握情况，但手动出题耗时耗力。本文对三种主流大模型——Llama 2、Mistral 和 GPT-3.5——进行对比分析，探究其生成高质量、有挑战性的多选题的可行性。我们不依赖模型自带知识，而是通过在提示中嵌入知识内容，控制测试来源并抑制幻觉。21位教育工作者参与实验表明，GPT-3.5在多项指标上表现最优。同时，研究也揭示了教育领域对AI应用仍存在一定的接受阻力。该研究为大模型生成教育试题提供了实证支持，也为未来教育智能化发展提供关键洞见。
原文摘要 · Abstract (English)
Integrating Artificial Intelligence (AI) in educational settings has brought new learning approaches, transforming the practices of both students and educators. Among the various technologies driving this transformation, Large Language Models (LLMs) have emerged as powerful tools for creating educational materials and question answering, but there are still space for new applications. Educators commonly use Multiple-Choice Questions (MCQs) to assess student knowledge, but manually generating these questions is resource-intensive and requires significant time and cognitive effort. In our opinion, LLMs offer a promising solution to these challenges. This paper presents a novel comparative analysis of three widely known LLMs - Llama 2, Mistral, and GPT-3.5 - to explore their potential for creating informative and challenging MCQs. In our approach, we do not rely on the knowledge of the LLM, but we inject the knowledge into the prompt to contrast the hallucinations, giving the educators control over the test's source text, too. Our experiment involving 21 educators shows that GPT-3.5 generates the most effective MCQs across several known metrics. Additionally, it shows that there is still some reluctance to adopt AI in the educational field. This study sheds light on the potential of LLMs to generate MCQs and improve the educational experience, providing valuable insights for the future.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。