arXiv:2507.21049cs.LGcs.CV2025-07ICCV被引 5

通过分析任务表示的显著性,提升多任务学习中的知识互补性。

Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning

  • 基于表示层任务显著性,动态调节任务间交互
  • 在四个基准上实现竞争力性能,无需复杂权重调整
  • 适合需要稳定多任务训练的场景,如跨领域学习

尽管多任务学习能利用任务间的互补知识,但现有优化方法仍聚焦于通过优化器中心的损失缩放和梯度操作解决冲突,难以持续提升效果。本文提出,共享表示空间中蕴含丰富信息,可为任务间互补提供新途径,而这一方向尚未被充分探索。为此,我们提出Rep-MTL,通过表示层任务显著性量化任务特异性优化与共享表示学习间的交互。借助基于熵的惩罚与样本级跨任务对齐,该方法在不牺牲单任务有效训练的前提下,缓解负向迁移,显式促进互补信息共享。在涵盖任务偏移与域偏移的四个挑战性基准上进行实验,结果表明,即便采用基础等权策略,Rep-MTL仍实现有竞争力的性能提升,并具备良好效率。此外,幂律指数分析验证其在任务特异性学习与跨任务共享间的平衡能力。

原文摘要 · Abstract (English)

Despite the promise of Multi-Task Learning in leveraging complementary knowledge across tasks, existing multi-task optimization (MTO) techniques remain fixated on resolving conflicts via optimizer-centric loss scaling and gradient manipulation strategies, yet fail to deliver consistent gains. In this paper, we argue that the shared representation space, where task interactions naturally occur, offers rich information and potential for operations complementary to existing optimizers, especially for facilitating the inter-task complementarity, which is rarely explored in MTO. This intuition leads to Rep-MTL, which exploits the representation-level task saliency to quantify interactions between task-specific optimization and shared representation learning. By steering these saliencies through entropy-based penalization and sample-wise cross-task alignment, Rep-MTL aims to mitigate negative transfer by maintaining the effective training of individual tasks instead pure conflict-solving, while explicitly promoting complementary information sharing. Experiments are conducted on four challenging MTL benchmarks covering both task-shift and domain-shift scenarios. The results show that Rep-MTL, even paired with the basic equal weighting policy, achieves competitive performance gains with favorable efficiency. Beyond standard performance metrics, Power Law exponent analysis demonstrates Rep-MTL's efficacy in balancing task-specific learning and cross-task sharing. The project page is available at HERE.

多任务学习表示学习任务显著性知识互补

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。