用模型自身参数变化筛选数据,让大模型微调更高效且抗噪声。
DONOD: Efficient and Generalizable Instruction Fine-Tuning for LLMs via Model-Intrinsic Dataset Pruning
- 基于模型参数变化设计两个指标,自动识别无效和有害训练样本。
- 删掉70%数据后,特定领域准确率提升14.90%,跨域准确率提升5.67%。
- 方法不依赖额外模型,适配不同大小模型,适合工业级微调场景。
大语言模型的领域自适应微调广泛采用,但传统监督微调(SFT)常削弱跨域泛化能力,并受噪声数据影响。为此,我们提出DONOD,一种轻量级、基于模型内在参数的数据剪枝方法。该方法通过两个基于模型参数的指标评估数据:Δ权重范数(DON),衡量数据对模型权重的累积影响;权重变化范数(NOD),量化权重不稳定性。结合TOPSIS算法,无需依赖辅助模型即可有效过滤噪声、不可学习及损害泛化的样本。在数学任务实验中,经DONOD筛选的数据显著提升微调效率与鲁棒性。仅保留30%数据(剔除70%),目标域准确率提升14.90%,跨域准确率提升5.67%。且小模型(如Llama 3.1-8B)剪枝后的数据可有效泛化至大模型(如Llama 2-13B)。相比现有方法,DONOD保持数据无关性,适用范围更广,性能相当或更优。代码将公开。
原文摘要 · Abstract (English)
Ad-hoc instruction fine-tuning of large language models (LLMs) is widely adopted for domain-specific adaptation. While domain-specific supervised fine-tuning (SFT) is effective and efficient, it often weakens cross-domain generalization and struggles with noisy training data. To address these challenges, we propose DONOD, a lightweight model-intrinsic data pruning method. Our approach evaluates data using two model-parameter-based metrics: Delta of Norm (DON), which captures the cumulative influence on model weights, and Norm of Delta (NOD), which quantifies weight instability. Moreover, by employing the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) algorithm, we effectively filter noisy, unlearnable, and generalization-harming samples without relying on auxiliary models during the SFT process. Experiments on mathematical tasks demonstrate that data selected by DONOD achieves superior fine-tuning efficiency and improved robustness against noisy data. By filtering out 70% of the whole dataset, we improve target-domain accuracy by 14.90% and cross-domain accuracy by 5.67%. Meanwhile, our selected data present superior cross-architecture generalization. Data pruned by smaller models (e.g., Llama 3.1-8B) generalize effectively on larger models (e.g., Llama 2-13B). Compared to existing related methodologies, DONOD demonstrates comparable or superior performance while remaining dataset-agnostic, enabling broader applicability. Code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。