arXiv:2511.10903cs.CLcs.AI2025-11被引 6

用简单模型加数据增强,高效自动分类试题认知层次。

Automated Analysis of Learning Outcomes and Exam Questions Based on Bloom's Taxonomy

  • 采用增强版SVM处理小样本数据,效果优于复杂深度模型。
  • 传统ML中SVM达94%准确率,远超RNN与BERT等模型。
  • 大模型零样本表现尚可,适合无标注数据场景使用。

本文研究基于布卢姆认知分类的考试题目与学习目标自动分类。构建了一个包含600个句子的小型标注数据集,涵盖知识、理解、应用、分析、综合、评价六类认知层次。实验对比了朴素贝叶斯、逻辑回归、支持向量机(SVM)等传统机器学习模型,以及LSTM、BiLSTM、GRU、BiGRU等循环神经网络,还有BERT、RoBERTa等基于Transformer的模型,以及OpenAI、Gemini、Ollama、Anthropic等大语言模型。所有模型在不同预处理和数据增强策略(如同义词替换、词嵌入)下进行评估。结果显示,经过数据增强的SVM在准确率、召回率和F1分数上均达到94%,且过拟合轻微;而RNN模型和BERT出现严重过拟合,RoBERTa虽初期缓解但训练后期仍显过拟合。大语言模型零样本测试中,OpenAI与Gemini表现最佳,准确率约0.72–0.73,F1分数相近。研究揭示了在小数据下训练复杂深度模型的挑战,强调了精心设计的数据增强与简化算法(如增强SVM)在布卢姆分类任务中的价值。

原文摘要 · Abstract (English)

This paper explores the automatic classification of exam questions and learning outcomes according to Bloom's Taxonomy. A small dataset of 600 sentences labeled with six cognitive categories - Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation - was processed using traditional machine learning (ML) models (Naive Bayes, Logistic Regression, Support Vector Machines), recurrent neural network architectures (LSTM, BiLSTM, GRU, BiGRU), transformer-based models (BERT and RoBERTa), and large language models (OpenAI, Gemini, Ollama, Anthropic). Each model was evaluated under different preprocessing and augmentation strategies (for example, synonym replacement, word embeddings, etc.). Among traditional ML approaches, Support Vector Machines (SVM) with data augmentation achieved the best overall performance, reaching 94 percent accuracy, recall, and F1 scores with minimal overfitting. In contrast, the RNN models and BERT suffered from severe overfitting, while RoBERTa initially overcame it but began to show signs as training progressed. Finally, zero-shot evaluations of large language models (LLMs) indicated that OpenAI and Gemini performed best among the tested LLMs, achieving approximately 0.72-0.73 accuracy and comparable F1 scores. These findings highlight the challenges of training complex deep models on limited data and underscore the value of careful data augmentation and simpler algorithms (such as augmented SVM) for Bloom's Taxonomy classification.

教育分析认知分类小样本学习SVM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。