用散度引导模型合并，多任务表现更稳更强。
DivMerge: A divergence-based model merging method for multi-tasking
- 基于JS散度自动平衡任务重要性
- 无需额外标注数据，多任务性能更优
- 扩展性强，任务越多越稳定
多任务学习通常通过合并数据集来实现，但随着微调模型的增多，基于任务算术的模型合并成为新方法。当前主要挑战是任务干扰随任务数量增加而加剧。本文提出一种新方法，将不同任务训练的模型合并为单一模型，同时保持各任务强性能。该方法利用Jensen-Shannon散度指导合并过程，无需额外标注数据，并自动调节任务权重。相比现有方法，本方法在任务数量增加时仍具鲁棒性，且在多个基准上持续优于已有工作。
原文摘要 · Abstract (English)
Multi-task learning (MTL) is often achieved by merging datasets before fine-tuning, but the growing availability of fine-tuned models has led to new approaches such as model merging via task arithmetic. A major challenge in this setting is task interference, which worsens as the number of tasks increases. We propose a method that merges models trained on different tasks into a single model, maintaining strong performance across all tasks. Our approach leverages Jensen-Shannon divergence to guide the merging process without requiring additional labelled data, and automatically balances task importance. Unlike existing methods, our approach remains robust as the number of tasks grows and consistently outperforms prior work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。