arXiv:2504.07994cs.CLcs.AI2025-04中稿 · the 28th European …

评估本体对自动出题任务的适配性,提升教育问答质量

Evaluating the Fitness of Ontologies for the Task of Question Generation

  • 基于专家评估构建针对出题任务的本体评价指标体系
  • 实证发现不同本体在出题效果上差异显著,性能受结构特征影响
  • 为教育领域自动出题系统提供可量化的本体选型依据

基于本体的自动出题是语义感知系统的重要应用,可为多样教学环境生成大规模题库。其生成问题的质量与认知难度高度依赖底层本体的质量与建模方式,因此评估本体对出题任务的适配性至关重要。目前尚无针对本体特性如何影响出题过程的系统研究。本文提出一套面向教育场景下出题任务的本体适配性要求与专用评估指标,采用ROMEO方法论(一种用于识别任务特定指标的结构化框架),通过专家对生成问题的评估推导出评价指标。为验证指标有效性,将它们应用于先前用于出题的多个本体,结果显示指标得分与已有研究结论一致且互补。分析表明,本体特性显著影响出题效果,不同本体表现各异,强调了针对自动出题任务评估本体质量的重要性。

原文摘要 · Abstract (English)

Ontology-based question generation is an important application of semantic-aware systems that enables the creation of large question banks for diverse learning environments. The effectiveness of these systems, both in terms of the calibre and cognitive difficulty of the resulting questions, depends heavily on the quality and modelling approach of the underlying ontologies, making it crucial to assess their fitness for this task. To date, there has been no comprehensive investigation into the specific ontology aspects or characteristics that affect the question generation process. Therefore, this paper proposes a set of requirements and task-specific metrics for evaluating the fitness of ontologies for question generation tasks in pedagogical settings. Using the ROMEO methodology (a structured framework used for identifying task-specific metrics), a set of evaluation metrics have been derived from an expert assessment of questions generated by a question generation model. To validate the proposed metrics, we apply them to a set of ontologies previously used in question generation to illustrate how the metric scores align with and complement findings reported in earlier studies. The analysis confirms that ontology characteristics significantly impact the effectiveness of question generation, with different ontologies exhibiting varying performance levels. This highlights the importance of assessing ontology quality with respect to Automatic Question Generation (AQG) tasks.

本体评估自动出题教育AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。