提出新方法缓解多任务学习中的任务不平衡问题，提升模型整体性能。
Injecting Imbalance Sensitivity for Multi-Task Learning
- 通过约束梯度投影范数，增强模型对任务不平衡的敏感性。
- 在多个主流多任务基准上表现优于现有方法，尤其在任务差异大时优势明显。
- 适合需要平衡多个异构任务的深度学习应用场景。
多任务学习（MTL）已成为部署深度学习模型于实际应用的有前景方法。近期研究提出了基于优化的学习范式，以建立任务共享表征。然而，本文通过实证分析指出，这些研究——尤其是基于梯度的方法——主要关注任务冲突问题，而忽略了任务不平衡/主导性可能带来的更大影响。基于此视角，我们通过在梯度投影范数上施加约束，增强了现有基线方法对不平衡的敏感性。为验证所提IMbalance-sensitive Gradient（IMGrad）下降方法的有效性，我们在多个主流多任务基准上进行了评估，涵盖监督学习任务和强化学习任务。实验结果一致表明其具有竞争力的性能。
原文摘要 · Abstract (English)
Multi-task learning (MTL) has emerged as a promising approach for deploying deep learning models in real-life applications. Recent studies have proposed optimization-based learning paradigms to establish task-shared representations in MTL. However, our paper empirically argues that these studies, specifically gradient-based ones, primarily emphasize the conflict issue while neglecting the potentially more significant impact of imbalance/dominance in MTL. In line with this perspective, we enhance the existing baseline method by injecting imbalance-sensitivity through the imposition of constraints on the projected norms. To demonstrate the effectiveness of our proposed IMbalance-sensitive Gradient (IMGrad) descent method, we evaluate it on multiple mainstream MTL benchmarks, encompassing supervised learning tasks as well as reinforcement learning. The experimental results consistently demonstrate competitive performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。