修复知识追踪中因项目偏差导致的预测能力下降问题
Recovering Stranded Discrimination in Knowledge Tracing: Per-Item Bias Correction via Empirical-Bayes Shrinkage

- 通过状态空间建模与经验贝叶斯收缩,对每个题目进行独立校正
- 在四个数据集上全部提升AUC,稀疏题目表现更优
- 适用于教育外领域,当模型存在实体级偏差时同样有效
部署的知识追踪模型通常训练后固定不变,但因骨干架构对单个题目的表达能力有限,以及部署后题目属性变化,导致系统性题目级对数几率偏差,降低预测质量。全局后处理校准器如Platt缩放、温度缩放和等倾回归虽能改善概率估计,但无法提升判别能力(以AUC衡量),这是单调分数变换的结构性结果;恢复被滞留的判别力需依赖题目身份信息。本文提出SLC(状态空间对数校正):将二分类观测转换为高斯伪观测(通过拉普拉斯/迭代重加权最小二乘法),利用卡尔曼平滑器实施经验贝叶斯收缩,并拟合偏移-Platt链接。状态空间形式还导出可检测性下界,解释了在当前数据密度下时间追踪为何无效。在四个数据集、五种骨干网络、三个随机种子下,SLC在所有四组数据上均提升AUC,三组提升负对数似然,优势集中于稀疏题目。跨域对照表明,该现象在教育之外亦可能发生,只要部署的骨干模型留下实体级偏差。
原文摘要 · Abstract (English)
Deployed knowledge-tracing models are typically frozen after training, yet systematic per-item logit bias arises, from limited per-item expressivity in backbone architectures and from post-deployment shifts in item properties, degrading prediction quality. Global post-hoc calibrators such as Platt scaling, temperature scaling, and isotonic regression improve probability estimates but leave discriminative ability, as measured by AUC, unchanged. This AUC invariance is a structural consequence of monotone score-only transforms; recovering the stranded discrimination requires conditioning on item identity. We propose SLC (State-space Logit Correction), which converts binary observations to Gaussian pseudo-observations via Laplace/IRLS, applies empirical-Bayes shrinkage through a Kalman smoother, and fits an offset-Platt link. The state-space formulation also yields a detectability bound that characterizes the Bernoulli information floor, explaining why temporal tracking provides no benefit at current data densities. Across four datasets, five backbones, and three seeds, SLC improves AUC on all four datasets and NLL on three, with the advantage concentrating on sparse items. Cross-domain controls suggest that the same phenomenon can arise beyond education when the deployed backbone leaves entity-level bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。