arXiv:2412.00125cs.CL2024-12

用知识注入提升教育内容检索效率,小模型也能精准回答专业问题。

Efficient Learning Content Retrieval with Knowledge Injection

  • 用QLoRA微调Phi-2/Phi-3,结合RAG系统增强问答能力。
  • 在华为人才平台数据上训练,支持多级计算机科学课程查询。
  • 通过多项指标评估,Φ-2+RAG在准确率与召回率上表现最佳。

随着在线教育平台发展,跨领域教育资源日益丰富,尤其在涉及众多关联知识点的ICT领域,学习者难以高效定位合适内容。本文提出一种轻量级领域专用聊天机器人,基于Phi-2和Phi-3语言模型,采用QLoRA方法进行微调,训练数据来自华为人才平台中不同层级的计算机科学课程。通过从JSON、PPT、DOC等格式中提取420组问答对,构建向量数据库并接入RAG系统,实现知识增强。使用500组问答对对模型进行微调。最终通过ROUGE、BERTScore、METEOR、BLEU等13项指标评估,Φ-2模型在RAG支持下达到精确率为0.84,F1得分为0.82,综合表现最优,适用于实际教学场景。

原文摘要 · Abstract (English)

With the rise of online education platforms, there is a growing abundance of educational content across various domain. It can be difficult to navigate the numerous available resources to find the most suitable training, especially in domains that include many interconnected areas, such as ICT. In this study, we propose a domain-specific chatbot application that requires limited resources, utilizing versions of the Phi language model to help learners with educational content. In the proposed method, Phi-2 and Phi-3 models were fine-tuned using QLoRA. The data required for fine-tuning was obtained from the Huawei Talent Platform, where courses are available at different levels of expertise in the field of computer science. RAG system was used to support the model, which was fine-tuned by 500 Q&A pairs. Additionally, a total of 420 Q&A pairs of content were extracted from different formats such as JSON, PPT, and DOC to create a vector database to be used in the RAG system. By using the fine-tuned model and RAG approach together, chatbots with different competencies were obtained. The questions and answers asked to the generated chatbots were saved separately and evaluated using ROUGE, BERTScore, METEOR, and BLEU metrics. The precision value of the Phi-2 model supported by RAG was 0.84 and the F1 score was 0.82. In addition to a total of 13 different evaluation metrics in 4 different categories, the answers of each model were compared with the created content and the most appropriate method was selected for real-life applications.

知识注入RAG教育AI小模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。