新方法让大模型微调时不丢通用能力，还能提升专业任务表现。
Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
- 重建基础模型的指令分布，生成高质量通用数据
- 合成数据与领域数据混合微调，有效缓解灾难性遗忘
- 无需原始数据，适合第三方开发者使用
监督微调（SFT）是提升大语言模型指令遵循能力并适配特定领域的关键步骤。然而，SFT常导致模型通用能力退化，即灾难性遗忘。该问题在第三方开发者微调开源模型时尤为严重，因原始SFT数据通常不可得。为此，我们提出一种新颖且低成本的SFT方法，可在不依赖原始SFT数据的情况下有效缓解灾难性遗忘。方法首先重建基础模型的可能指令分布，再通过多模型生成与过滤管道合成高质量通用数据集。该合成数据集与新领域数据混合用于微调。实验表明，该方法不仅保持了模型在通用领域的能力，还提升了特定任务性能，优于使用公开SFT数据集的基线方法。
原文摘要 · Abstract (English)
Supervised Fine-Tuning (SFT) is a critical step for enhancing the instruction-following capabilities of Large Language Models (LLMs) and adapting them to specialized domains. However, SFT often leads to a degradation of the model's general abilities, a phenomenon known as catastrophic forgetting. This problem is exacerbated when third-party practitioners fine-tune open-source models, as the original SFT data is typically not available. To address this challenge, we propose a novel and cost-effective SFT method that effectively mitigates catastrophic forgetting without requiring access to the original SFT data. Our approach first reconstructs the likely instruction distribution of the base model. It then employs a multi-model generation and filtering pipeline to synthesize a high-quality general-purpose dataset. This synthetic dataset is mixed with new, domain-specific data for fine-tuning. Experimental results show that our method not only preserves the model's capabilities in general domains but also improves task-specific performance, outperforming baselines that use publicly available SFT datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。