arXiv:2509.23948cs.LG2025-09

提出对损失缩放不变的多任务学习方法,解决任务主导问题。

Monotonic Transformation Invariant Multi-task Learning

  • 基于博弈论的梯度调整策略,避免任务损失缩放影响。
  • 理论证明非凸场景下可收敛到帕累托最优解。
  • 在损失尺度差异大时表现远超现有方法,适合复杂多任务场景。

多任务学习(MTL)通常依赖加权平均不同任务损失或梯度的方案,旨在通过启发式方法找到帕累托平稳点,但需访问任务损失值、梯度或两者。此类方法面临核心挑战:任务损失可任意缩放,导致某些任务主导训练,损害整体性能。近期合作博弈理论进展——方向性博弈解(DiBS)能抵御非仿射单调变换,实现帕累托平稳解。然而,其在非凸MTL设置下的收敛性尚不明确。本文证明,在标准假设下,DiBS迭代子序列在非凸任务损失下收敛至帕累托平稳点,并提出适用于MTL的改进版本DiBS-MTL,计算效率高于以往博弈启发方法。实验证明,DiBS-MTL在标准基准上与主流MTL方法相当,且在多个损失尺度严重失衡的案例中显著优于当前最佳基线,凸显了对损失景观单调非仿射变换不变性的重要性。代码见https://github.com/suryakmurthy/dibs-mtl。

原文摘要 · Abstract (English)

Multi-task learning (MTL) algorithms typically rely on schemes that combine different task losses or their gradients through weighted averaging. These methods aim to find Pareto stationary points by using heuristics that require access to task loss values, gradients, or both. In doing so, a central challenge arises because task losses can be arbitrarily scaled relative to one another, causing certain tasks to dominate training and degrade overall performance. A recent advance in cooperative bargaining theory, the Direction-based Bargaining Solution (DiBS), yields Pareto stationary solutions immune to task domination because of its invariance to monotonic nonaffine task loss transformations. However, the convergence behavior of DiBS in nonconvex MTL settings is currently not understood. To this end, we prove that under standard assumptions, a subsequence of DiBS iterates converges to a Pareto stationary point when task losses are nonconvex, and propose DiBS-MTL, an adaptation of DiBS to the MTL setting which is more computationally efficient that prior bargaining-inspired MTL approaches. Finally, we empirically show that DiBS-MTL is competitive with leading MTL methods on standard benchmarks, and it drastically outperforms state-of-the-art baselines in multiple examples with poorly-scaled task losses, highlighting the importance of invariance to nonaffine monotonic transformations of the loss landscape. Code available at https://github.com/suryakmurthy/dibs-mtl

多任务学习博弈论优化算法损失归一化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。