arXiv:2505.17470cs.CLcs.AI2025-05被引 1

让大模型自己挑出新知识,高效适配特定领域。

SLearnLLM: A Self-Learning Framework for Efficient Domain-Specific Adaptation of Large Language Models

  • 模型自测答题并筛选错误答案,只用新知识微调。
  • 农业与医学实验中训练时间大幅减少,效果相当。
  • 适合资源有限但需快速领域适配的场景。

在使用监督微调(SFT)将大语言模型(LLMs)适配特定领域时,一个关键挑战是:是否应使用全部SFT数据进行微调?由于缺乏对模型过往训练数据的了解,通常直接使用完整数据集。然而,若SFT数据与模型已有知识高度重叠,性能提升有限,造成计算资源浪费。识别SFT数据中模型未知的知识并以此微调,可显著提升训练效率。为此,我们提出一种受人类学习模式启发的自学习框架。该框架以特定领域的SFT数据为输入,首先让模型回答数据中的问题,再客观评估回答并过滤掉错误的问答对,最后基于筛选后的数据集进行微调。农业与医学领域的实验表明,该方法显著减少训练时间,同时达到与全量数据微调相当的效果。通过聚焦于SFT数据中的未知知识,本方法有效提升了大模型微调的效率。

原文摘要 · Abstract (English)

When using supervised fine-tuning (SFT) to adapt large language models (LLMs) to specific domains, a significant challenge arises: should we use the entire SFT dataset for fine-tuning? Common practice often involves fine-tuning directly on the entire dataset due to limited information on the LLM's past training data. However, if the SFT dataset largely overlaps with the model's existing knowledge, the performance gains are minimal, leading to wasted computational resources. Identifying the unknown knowledge within the SFT dataset and using it to fine-tune the model could substantially improve the training efficiency. To address this challenge, we propose a self-learning framework for LLMs inspired by human learning pattern. This framework takes a fine-tuning (SFT) dataset in a specific domain as input. First, the LLMs answer the questions in the SFT dataset. The LLMs then objectively grade the responses and filter out the incorrectly answered QA pairs. Finally, we fine-tune the LLMs based on this filtered QA set. Experimental results in the fields of agriculture and medicine demonstrate that our method substantially reduces training time while achieving comparable improvements to those attained with full dataset fine-tuning. By concentrating on the unknown knowledge within the SFT dataset, our approach enhances the efficiency of fine-tuning LLMs.

大模型微调自学习领域适配效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。