揭示任务向量与梯度的等价关系,解释为何模型合并有效
On Task Vectors and Gradients
- 任务向量等于负梯度,学习率决定缩放比例
- 单轮微调的梯度主导整个微调轨迹,误差可量化
- 仅单轮微调即可实现接近全收敛模型的合并效果
任务算术已成为一种简单而强大的模型合并技术,可将多个微调模型整合为一个。尽管其在实践中表现优异,但缺乏清晰的理论解释。本文通过建立任务向量与任务损失梯度之间的联系,为任务算术提供了严格的理论基础。我们证明,在标准梯度下降下,经过一轮微调生成的任务向量恰好等于负梯度,缩放因子为学习率。在多轮实际设置中,该等价关系近似成立,误差项为二阶,且对前馈网络给出了显式上界。七项视觉基准的实证分析验证了该理论,表明第一轮梯度在范数和方向上均主导微调轨迹。关键启示是:仅用单轮微调的模型进行合并,性能常可媲美完全收敛模型。这些发现将任务算术重新理解为一种近似多任务学习,阐明了其有效性,并凸显了早期训练动态在模型合并中的核心作用。
原文摘要 · Abstract (English)
Task arithmetic has emerged as a simple yet powerful technique for model merging, enabling the combination of multiple finetuned models into one. Despite its empirical success, a clear theoretical explanation of why and when it works is lacking. This paper provides a rigorous theoretical foundation for task arithmetic by establishing a connection between task vectors and gradients of the task losses. We show that under standard gradient descent, a task vector generated from one epoch of finetuning is exactly equivalent to the negative gradient of the loss, scaled by the learning rate. For the practical multi-epoch setting, we prove that this equivalence holds approximately, with a second-order error term that we explicitly bound for feed-forward networks. Our empirical analysis across seven vision benchmarks corroborates our theory, demonstrating that the first-epoch gradient dominates the finetuning trajectory in both norm and direction. A key implication is that merging models finetuned for only a single epoch often yields performance comparable to merging fully converged models. These findings reframe task arithmetic as a form of approximate multitask learning, providing a clear rationale for its effectiveness and highlighting the critical role of early training dynamics in model merging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。