arXiv:2411.18823cs.CV2024-11被引 4

用分层任务令牌自动发现像素级标签,提升部分标注下的多任务密集预测性能。

Multi-Task Label Discovery via Hierarchical Task Tokens for Partially Annotated Dense Predictions

  • 设计全局与细粒度层级任务令牌,实现跨任务特征交互与像素级监督信号生成。
  • 在NYUD-v2、Cityscapes等数据集上显著优于现有最先进方法,提升明显。
  • 适合研究多任务学习中标签稀疏场景的开发者,尤其关注密集预测任务。

近年来,同时学习多个部分标注的密集预测任务成为重要研究方向。以往方法主要依赖跨任务关系或对抗训练进行正则化,虽取得良好效果,但仍面临缺乏直接像素级监督和额外训练重型映射网络的问题。为此,本文提出一种新方法,通过优化一组紧凑可学习的分层任务令牌(包括全局与细粒度令牌),在特征与预测层面发现一致的像素级监督信号。全局任务令牌用于实现全局上下文中的跨任务特征交互;每个任务的细粒度空间令牌由对应全局令牌学习而来,并与任务特异性特征图进行密集交互。所学全局与局部令牌进一步用于不同粒度级别上发现伪任务特定密集标签,可直接监督多任务密集预测框架的学习。在挑战性数据集NYUD-v2、Cityscapes和PASCAL Context上的大量实验表明,该方法显著优于现有最先进方法。

原文摘要 · Abstract (English)

In recent years, simultaneous learning of multiple dense prediction tasks with partially annotated label data has emerged as an important research area. Previous works primarily focus on leveraging cross-task relations or conducting adversarial training for extra regularization, which achieve promising performance improvements, while still suffering from the lack of direct pixel-wise supervision and extra training of heavy mapping networks. To effectively tackle this challenge, we propose a novel approach to optimize a set of compact learnable hierarchical task tokens, including global and fine-grained ones, to discover consistent pixel-wise supervision signals in both feature and prediction levels. Specifically, the global task tokens are designed for effective cross-task feature interactions in a global context. Then, a group of fine-grained task-specific spatial tokens for each task is learned from the corresponding global task tokens. It is embedded to have dense interactions with each task-specific feature map. The learned global and local fine-grained task tokens are further used to discover pseudo task-specific dense labels at different levels of granularity, and they can be utilized to directly supervise the learning of the multi-task dense prediction framework. Extensive experimental results on challenging NYUD-v2, Cityscapes, and PASCAL Context datasets demonstrate significant improvements over existing state-of-the-art methods for partially annotated multi-task dense prediction.

多任务学习密集预测标签发现自监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。