用指令模型引导微调，让大模型在专精任务上表现更好且不丢通用能力。
GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
- 用指令模型的置信度信号指导低秩适配器训练
- 在数学与知识类任务上显著优于直接微调和基线方法
- 适合需要保持通用能力又想专精特定任务的场景
一种有前景的指令微调语言模型适应范式是，在预训练基础模型上学习任务特定更新，随后将其合并到指令微调模型中。然而，现有方法通常将指令微调模型视为被动目标，仅在最终合并阶段参与，未在训练过程中提供指导。我们提出 GIFT（Guided Fine-Tuning and Transfer），一个简单高效的框架，将指令模型的指导融入任务适应过程。GIFT 使用来自指令微调模型的置信度信号，在预训练基础模型上微调一个低秩适配器。学习到的适配器随后被合并到指令微调模型中，生成保留通用指令遵循行为的任务专用模型。我们在多个模型家族和规模下，对数学与知识密集型基准进行了评估。结果表明，GIFT 在各项任务中均持续优于直接微调及代表性基于迁移的基线方法，同时保持鲁棒泛化性和有利的测试时缩放行为。
原文摘要 · Abstract (English)
A promising paradigm for adapting instruction-tuned language models is to learn task-specific updates on a pretrained base model and subsequently merge them into the instruction-tuned model. However, existing approaches typically treat the instruction-tuned model as a passive target that is only involved at the final merging stage, without guiding the training process. We propose GIFT (Guided Fine-Tuning and Transfer), a simple and efficient framework that incorporates guidance from the instruction model into task adaptation. GIFT fine-tunes a low-rank adapter on the pretrained base model using confidence signals derived from the instruction-tuned model. The learned adapter is then merged into the instruction-tuned model, yielding task-specialized models that preserve general instruction-following behavior. We evaluate GIFT on mathematical and knowledge-intensive benchmarks across multiple model families and scales. Results show that GIFT consistently outperforms direct fine-tuning and representative transfer-based baselines, while maintaining robust generalization and favorable test-time scaling behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。