arXiv:2507.07883cs.LG2025-07ICCV被引 3

提出轻量级方法SAMO,缓解多任务学习中的梯度冲突问题。

SAMO: A Lightweight Sharpness-Aware Approach for Multi-Task Optimization with Joint Global-Local Perturbation

  • 采用全局-局部联合扰动机制,提升优化稳定性。
  • 仅需前向传播近似局部扰动,显著降低计算开销。
  • 在多个基准上验证有效且高效,适合资源受限场景。

多任务学习(MTL)通过共享模型捕捉多任务间的共性,降低计算成本并提升数据效率。然而,任务间梯度方向或幅度差异导致的冲突限制了模型性能。尖锐度感知最小化(SAM)能同时最小化任务损失并降低损失曲面的尖锐度,实验表明其可有效缓解多任务学习中的冲突。受此启发,我们探索将SAM引入MTL,但面临两大挑战:如何融合平均损失梯度(全局)与各任务梯度(局部)尚不明确;直接计算每任务梯度带来巨大计算与内存开销。为此,我们提出SAMO——一种轻量级尖锐度感知多任务优化方法,采用联合全局-局部扰动策略。局部扰动通过仅前向传播近似,并进行逐层归一化以提升效率。在多个多任务基准上的大量实验验证了该方法的有效性与高效性。代码已开源。

原文摘要 · Abstract (English)

Multi-task learning (MTL) enables a joint model to capture commonalities across multiple tasks, reducing computation costs and improving data efficiency. However, a major challenge in MTL optimization is task conflicts, where the task gradients differ in direction or magnitude, limiting model performance compared to single-task counterparts. Sharpness-aware minimization (SAM) minimizes task loss while simultaneously reducing the sharpness of the loss landscape. Our empirical observations show that SAM effectively mitigates task conflicts in MTL. Motivated by these findings, we explore integrating SAM into MTL but face two key challenges. While both the average loss gradient and individual task gradients-referred to as global and local information-contribute to SAM, how to combine them remains unclear. Moreover, directly computing each task gradient introduces significant computational and memory overheads. To address these challenges, we propose SAMO, a lightweight \textbf{S}harpness-\textbf{A}ware \textbf{M}ulti-task \textbf{O}ptimization approach, that leverages a joint global-local perturbation. The local perturbations are approximated using only forward passes and are layerwise normalized to improve efficiency. Extensive experiments on a suite of multi-task benchmarks demonstrate both the effectiveness and efficiency of our method. Code is available at https://github.com/OptMN-Lab/SAMO.

多任务学习优化算法轻量化梯度冲突

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。