小模型通过生成再验证，高效产出符合教学目标的优质问题
Generate-Then-Validate: A Novel Question Generation Approach Using Small Language Models
- 采用先大量生成再基于概率推理筛选的双阶段策略
- 人工与大模型评估均认可生成问题有明确答案且贴合教学目标
- 适合教育AI、智能题库等需要低成本高质量题目的场景
我们探索了小型语言模型（SLMs）在自动问题生成中的应用，作为学习分析研究中主流大型模型的补充。提出一种新型问题生成流程，利用SLMs的文本生成与概率推理能力，生成高质量问题。该流程采用“生成-验证”策略：首先进行大规模生成以产生丰富候选问题，并基于新颖的概率推理进行选择性验证。通过两项评估研究——一组七名专家与一个大型语言模型（LLM）——评估生成问题的质量。多数评判者（人类或LLM）认为生成的问题具有清晰答案，且总体上与预期学习目标高度一致。研究结果表明，只要设计合理的流程，小模型也能有效生成高质量问题。
原文摘要 · Abstract (English)
We explore the use of small language models (SLMs) for automatic question generation as a complement to the prevalent use of their large counterparts in learning analytics research. We present a novel question generation pipeline that leverages both the text generation and the probabilistic reasoning abilities of SLMs to generate high-quality questions. Adopting a "generate-then-validate" strategy, our pipeline first performs expansive generation to create an abundance of candidate questions and refine them through selective validation based on novel probabilistic reasoning. We conducted two evaluation studies, one with seven human experts and the other with a large language model (LLM), to assess the quality of the generated questions. Most judges (humans or LLMs) agreed that the generated questions had clear answers and generally aligned well with the intended learning objectives. Our findings suggest that an SLM can effectively generate high-quality questions when guided by a well-designed pipeline that leverages its strengths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。