用大模型知识蒸馏出高效问答模拟器,提升教育推荐系统训练效果。
Constructing a Question-Answering Simulator through the Distillation of LLMs
- 从大模型中蒸馏知识和推理能力,构建轻量级问答模拟器。
- 在问答模拟与知识追踪任务上均达到优秀性能,优于传统方法。
- 适合需要高效模拟学生行为的教育AI系统开发者使用。
问答模拟器是一种模仿真实学生学习行为并预测其答题正确性的模型。它使教育推荐系统(ERS)无需与真实学生交互即可生成大量训练数据,从而避免因模型未充分训练而产生有害推荐,影响实际学习。现有方法分为两类:(1)不依赖大模型的方法,使用传统序列模型将问答历史转化为向量表示后进行预测,推理速度快但性能较差;(2)基于大模型的方法,利用大模型的领域知识和推理能力提升预测效果,性能更优但推理慢、显存消耗高。本文提出一种名为基于大模型蒸馏的模拟器(LDSim)的方法,通过从大模型中蒸馏领域知识和推理能力,提升预测性能。大量实验表明,LDSim在问答模拟和知识追踪(KT)任务上均表现优异。代码已公开于 https://anonymous.4open.science/r/LDSim-05A9。
原文摘要 · Abstract (English)
The question-answering (QA) simulator is a model that mimics real student learning behaviors and predicts their correctness of their responses to questions. QA simulators enable educational recommender systems (ERS) to collect large amounts of training data without interacting with real students, thereby preventing harmful recommendations made by an undertrained ERS from undermining actual student learning. Given the QA history, there are two categories of solutions to predict the correctness, conducting the simulation: (1) LLM-free methods, which apply a traditional sequential model to transfer the QA history into a vector representation first, and make predictions based on the representation; (2) LLM-based methods, which leverage the domain knowledge and reasoning capability of LLM to enhence the prediction. LLM-free methods offer fast inference but generally yield suboptimal performance. In contrast, most LLM-based methods achieve better results, but at the cost of slower inference speed and higher GPU memory consumption. In this paper, we propose a method named LLM Distillation based Simulator (LDSim), which distills domain knowledge and reasoning capability from an LLM to better assist prediction, thereby improving simulation performance. Extensive experiments demonstrate that our LDSim achieves strong results on both the simulation task and the knowledge tracing (KT) task. Our code is publicly available at https://anonymous.4open.science/r/LDSim-05A9.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。