arXiv:2410.12872cs.CLcs.AI2024-10EMNLP被引 1

用语义权重优化答题选项,让大模型更高效地追踪学习者知识状态。

Not All Options Are Created Equal: Textual Option Weighting for Token-Efficient LLM-Based Knowledge Tracing

  • 将学习者答题选项转化为语义标签,构建可解释的上下文表示
  • 在有限令牌下仍保持高性能,冷启动与热启动均优于现有方法
  • 适合需要低成本、高可扩展性的智能教育系统使用

大型语言模型(LLM)凭借强大的推理和泛化能力,成为知识追踪(KT)的有力工具。然而,现有基于LLM的KT方法在上下文学习中难以完整编码学习者的历史交互信息,导致可扩展性差且计算成本高。本文提出一种简单有效的框架——基于选项加权的知识追踪(LOKT),将学习者的答题历史编码为文本类别权重(TCOW),即为学习者选择的选项赋予语义标签(如“不足”),提升LLM可解释性。在多个多选题数据集上的实验表明,LOKT在冷启动和热启动设置下均优于现有非LLM及基于LLM的KT模型。此外,该方法支持高可扩展性和低计算开销,在严格令牌约束下仍表现优异。代码已公开。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have recently emerged as promising tools for knowledge tracing (KT) due to their strong reasoning and generalization abilities. While recent LLM-based KT methods have proposed new prompt formats, they struggle to represent the full interaction histories of example learners within a single prompt during in-context learning (ICL), resulting in limited scalability and high computational cost under token constraints. In this work, we present \textit{LLM-based Option-weighted Knowledge Tracing (LOKT)}, a simple yet effective framework that encodes the interaction histories of example learners in context as \textit{textual categorical option weights (TCOW)}. TCOW are semantic labels (e.g., ``inadequate'') assigned to the options selected by learners when answering questions, enhancing the interpretability of LLMs. Experiments on multiple-choice datasets show that LOKT outperforms existing non-LLM and LLM-based KT models in both cold-start and warm-start settings. Moreover, LOKT enables scalable and cost-efficient inference, achieving strong performance even under strict token constraints. Our code is available at \href{https://anonymous.4open.science/r/LOKT_model-3233}{https://anonymous.4open.science/r/LOKT\_model-3233}.

知识追踪大模型应用提示工程教育AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。