评测大模型生成教育题目的思维深度，发现优化提示词可提升高阶思维题目比例。
From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions
- 设计细粒度提示策略，减少重复并提升高阶思维题目生成率。
- 量化显示InternLM3在多层级思维转换上表现最优，认知跃迁更强。
- 揭示思维链提示的可解释性规律，适合教育AI研发与教学应用。
尽管大语言模型在自动化教育内容生成方面展现出潜力，但其生成能激发高阶思维问题的能力仍缺乏研究。本工作基于布卢姆分类法，评估六种广泛使用的LLM，考察其超越机械记忆、实现认知跃迁的能力。通过人机混合评估协议，在计算机科学、K-12数学与社会科学领域生成并分析了20,700个问题。主要贡献包括：(1) 一种细粒度提示策略，使Qwen2.5-7B-Instruct的问题重复率降低24.45%，InternLM3-8B-Instruct的高阶认知层级输出比例提升11.53%；(2) 提出认知跃迁强度（CogShift）与类别漂移的量化指标，揭示InternLM3在多层级转换中表现更优；(3) 可解释性分析发现指标层面相关性，增强思维链提示的透明度。研究强调认知感知提示设计的重要性，并为个性化学习系统部署提供基准。
原文摘要 · Abstract (English)
While LLMs show promise in automating educational content creation, their ability to generate questions that stimulate higher-order thinking remains understudied. This work evaluates six widely-used LLMs through a Bloom's Taxonomy lens, focusing on their capacity to transcend rote memorization and achieve cognitive leaps. Using a hybrid human--AI evaluation protocol, we generate and analyze 20{,}700 questions across computer science, K--12 math, and social-science domains. Key contributions include: (1) a fine-grained prompting strategy that reduces question repetitiveness by 24.45\% for Qwen2.5-7B-Instruct, and increases the proportion of higher-order cognitive level outputs by 11.53\% for InternLM3-8B-Instruct; (2) quantitative metrics for cognitive shift intensity (CogShift) and category drift, revealing InternLM3's superior performance in multi-level transitions; (3) an interpretability analysis revealing metric-level correlations that enhance the transparency of Chain-of-Thought prompting. Our findings highlight the importance of cognitive-aware prompt design and provide benchmarks for deploying LLMs in personalized learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。