arXiv:2512.06629cs.AIcs.IT2025-12被引 2

用轻量注入机制让扁平Transformer实现顶尖认知追踪效果

FlatFormer: A Flat Transformer Knowledge Tracing Model Based on Cognitive Bias Injection

  • 用注入式设计替代深层结构,降低模型复杂度
  • 在EdNet上比最强基线高8.3%AUC,参数少于15%
  • 适合需要实时推理的教育AI系统部署

知识追踪模型面临性能与复杂度的困境:捕捉学习会话和记忆衰退等认知动态通常需深层分层结构,但计算开销大,难以实时部署。为此,本文提出FlatFormer,一种基于“信息注入优于结构堆叠”新范式的简化架构。不同于参数庞大的分层模型,FlatFormer采用标准扁平Transformer,结合两项轻量级注入机制:(i) 混合输入编码,融合可学习会话标识符与固定正弦步嵌入;(ii) 预计算幂律偏置直接注入注意力逻辑,显式建模遗忘曲线。在四个大规模数据集(如EdNet、Junyi)上的实验表明,FlatFormer达到当前最优性能。例如,在EdNet数据集上,相比最强分层基线HiTSKT,其绝对AUC提升8.3%,参数量不足15%,推理速度约快三倍。结果验证了高认知保真度无需依赖复杂架构。

原文摘要 · Abstract (English)

Knowledge Tracing (KT) models face a critical ``Performance-Complexity Trap'': capturing complex cognitive dynamics like learning sessions and memory decay typically requires deep hierarchical architectures, which incur prohibitive computational costs for real-time deployment. To resolve this, we propose FlatFormer, a streamlined architecture based on the novel design paradigm of ``Information Injection over Structural Stacking.'' Unlike parameter-heavy hierarchical models, FlatFormer leverages a standard flat Transformer augmented with two lightweight injection mechanisms: (i) a hybrid input encoding strategy combining learnable session identifiers with fixed sinusoidal step embeddings; and (ii) a pre-computed power-law bias integrated directly into attention logits to explicitly model the forgetting curve. Extensive experiments on four large-scale datasets (e.g., EdNet, Junyi) show that FlatFormer achieves state-of-the-art performance. For example, on the EdNet dataset, compared to the strongest hierarchical baseline (HiTSKT), its absolute AUC increased by 8.3%, while using less than 15% of parameters, and inference speed was about three times faster. These results validate that high cognitive fidelity does not necessitate architectural complexity.

知识追踪扁平模型认知偏差Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。