通过分离任务专属参数区,解决大模型微调时的冲突问题。
DPI: Exploiting Parameter Heterogeneity for Interference-Free Fine-Tuning
- 识别每个任务的核心参数区域并动态隔离
- 多阶段微调中冻结已有任务参数,避免干扰
- 在多个数据集上显著提升多任务性能
监督微调(SFT)是将大语言模型适配到下游任务的关键步骤。然而,不同SFT任务间的目标冲突常引发“跷跷板效应”:优化一个任务可能损害其他任务表现,尤其当参数被随意更新时。本文提出一种基于参数异质性的解耦方法,假设跨任务干扰源于参数异质性。首先在多个SFT任务上独立微调,识别每个任务核心参数区域(即更新幅度最大的参数子集)。对核心区域高度重叠的任务进行合并联合训练,对不重叠的任务则分阶段处理。在多阶段SFT中,先前任务的核心参数被冻结,防止后续任务覆盖。在多个公开数据集上的实验证明,该动态参数隔离策略有效缓解数据冲突,在性能上持续优于多阶段和多任务微调基线。
原文摘要 · Abstract (English)
Supervised fine-tuning (SFT) is a crucial step for adapting large language models (LLMs) to downstream tasks. However, conflicting objectives across heterogeneous SFT tasks often induce the "seesaw effect": optimizing for one task may degrade performance on others, particularly when model parameters are updated indiscriminately. In this paper, we propose a principled approach to disentangle and isolate task-specific parameter regions, motivated by the hypothesis that parameter heterogeneity underlies cross-task interference. Specifically, we first independently fine-tune LLMs on diverse SFT tasks and identify each task's core parameter region as the subset of parameters exhibiting the largest updates. Tasks with highly overlapping core parameter regions are merged for joint training, while disjoint tasks are organized into different stages. During multi-stage SFT, core parameters acquired in prior tasks are frozen, thereby preventing overwriting by subsequent tasks. To verify the effectiveness of our method, we conducted intensive experiments on multiple public datasets. The results showed that our dynamic parameter isolation strategy consistently reduced data conflicts and achieved consistent performance improvements compared to multi-stage and multi-task tuning baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。