arXiv:2509.16959cs.LGcs.AI2025-09

用图着色动态分组任务,缓解多任务学习中的梯度冲突。

Graph Coloring for Multi-Task Learning

  • 构建梯度干扰图,通过贪心着色将任务分组。
  • 每轮仅更新同色任务,避免方向冲突导致的性能下降。
  • 理论保证收敛性,适合存在任务冲突的复杂场景。

多任务学习中,不同目标间存在冲突时,梯度会相互干扰并减缓收敛,可能降低模型最终性能。为此,我们提出 SON-GOKU 调度器:它计算梯度干扰,构建干扰图,并应用贪心图着色将任务划分为互不冲突的组。每个训练步仅激活一个组(颜色类)的任务,且随着训练过程动态重计算分组。通过确保每个小批量中仅包含方向一致的任务,该方法在不增加调参的前提下提升了现有优化器的有效性。由于组内任务更新方向兼容,多任务学习反而促进性能提升。六组不同数据集的实验表明,该干扰感知的图着色方法持续优于基线和当前最优多任务优化器。我们提供了充分的理论分析,证明分组与顺序更新可改善多任务学习,具备下降保证、收敛性及准确识别任务冲突或对齐的能力。

原文摘要 · Abstract (English)

When different objectives conflict with each other in multi-task learning, gradients begin to interfere and slow convergence, thereby potentially reducing the final model's performance. To address this, we introduce SON-GOKU, a scheduler that computes gradient interference, constructs an interference graph, and then applies greedy graph-coloring to partition tasks into groups that align well with each other. At each training step, only one group (color class) of tasks are activated, and the grouping partition is constantly recomputed as task relationships evolve throughout training. By ensuring that each mini-batch contains only tasks that pull the model in the same direction, our method improves the effectiveness of any underlying multi-task learning optimizer without additional tuning. Since tasks within these groups will update in compatible directions, multi-task learning will improve model performance rather than impede it. Empirical results on six different datasets show that this interference-aware graph-coloring approach consistently outperforms baselines and state-of-the-art multi-task optimizers. We provide extensive theory showing why grouping and sequential updates improve multi-task learning, with guarantees on descent, convergence, and the ability to accurately identify what tasks conflict or align.

多任务学习图着色梯度冲突优化器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。