构建统一评测框架,让习题推荐的两种方法能公平比较。
UniER: A Unified Benchmark for Item-level and Path-level Exercise Recommendation

- 设计统一指标WCG,兼容单题与路径推荐评估
- 9个数据集验证18种方法,发现路径推荐整体更优
- 开源代码助力可复现研究,适合教育技术开发者
个性化习题推荐能动态匹配学习者知识掌握情况,对满足现代教育中学生动态学习需求至关重要。当前该领域主要分为两类范式:单题推荐(ILER)关注即时状态转移,路径推荐(PLER)则注重构建连贯学习路径以实现累积收益最大化。尽管目标一致,但评价体系不同导致两者长期孤立,难以统一评测和公平对比。为此,本文提出统一习题推荐基准(UniER),涵盖9个数据集、4种生成方式,支持18种代表性方法的对比。通过多维度分析,结果表明PLER系统性优于ILER,尤其在极端稀疏和噪声环境下,单题推荐出现教学失效。我们还开源了UniER代码库,推动可复现研究,并展望未来方向。
原文摘要 · Abstract (English)
Personalized exercise recommendation dynamically aligns pedagogical resources with individual knowledge mastery, which is crucial for satisfying students' dynamic learning needs in modern education. The field is currently driven by two dominant paradigms: Item-Level Exercise Recommendation (ILER) optimizes for immediate single-step state transitions, while Path-Level Exercise Recommendation (PLER) constructs coherent learning paths to maximize cumulative gains. Despite sharing the same ultimate objective, disparate evaluation setups have kept these two lines of research isolated, hindering unified benchmarking and fair comparison. To fill the gap, in this paper, we present a Unified Benchmark for Exercise Recommendation (UniER), a comprehensive evaluation framework that unifies ILER and PLER. Specifically, we introduce Weighted Cognitive Gain (WCG) as a unified metric to measure cross-paradigm algorithmic performance. Our benchmark encompasses 9 datasets spanning four generation methods, facilitating the comparison of 18 representative ILER/PLER methods. Through multi-dimensional analyses covering effectiveness, generalizability, robustness, and efficiency, our results reveal the systematic dominance of PLER and expose the pedagogical failure of ILER's fragmented recommendations under extreme sparsity and noise. Furthermore, we provide an open-source codebase of UniER to foster reproducible research and outline potential directions for future investigations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。