提出梯度去冲突方法，让多任务学习更稳定高效。
Gradient Deconfliction via Orthogonal Projections onto Subspaces For Multi-task Learning
- 用子空间正交投影消除任务间梯度冲突
- 在多个数据集上实现多种权衡策略的顶尖性能
- 适合需要平衡多任务表现的研究者
尽管多任务学习（MTL）在众多实际场景中被广泛采用并取得成功，但其模型并不总能在所有任务上超越单任务模型，主要原因是任务间梯度冲突带来的负面影响。本文深入分析了梯度冲突的影响，强调获得非冲突梯度的重要性与优势，使任务间可采用简单而有效的权衡策略，并保持性能稳定。基于此，我们提出梯度去冲突方法GradOPS，通过将梯度正交投影到由其他任务特定梯度张成的子空间中，不仅完全解决任务间的梯度冲突，还能有效搜索不同权衡偏好下的多样化最优解。提供了收敛性理论分析，并在多个领域和多种基准上验证了算法性能。结果表明，该方法可在多个数据集上找到具有不同权衡策略的多个前沿解决方案。
原文摘要 · Abstract (English)
Although multi-task learning (MTL) has been a preferred approach and successfully applied in many real-world scenarios, MTL models are not guaranteed to outperform single-task models on all tasks mainly due to the negative effects of conflicting gradients among the tasks. In this paper, we fully examine the influence of conflicting gradients and further emphasize the importance and advantages of achieving non-conflicting gradients which allows simple but effective trade-off strategies among the tasks with stable performance. Based on our findings, we propose the Gradient Deconfliction via Orthogonal Projections onto Subspaces (GradOPS) spanned by other task-specific gradients. Our method not only solves all conflicts among the tasks, but can also effectively search for diverse solutions towards different trade-off preferences among the tasks. Theoretical analysis on convergence is provided, and performance of our algorithm is fully testified on multiple benchmarks in various domains. Results demonstrate that our method can effectively find multiple state-of-the-art solutions with different trade-off strategies among the tasks on multiple datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。