arXiv:2501.18509cs.CV2025-01

将复杂动作检测拆解为子任务,提升准确率与关系建模能力。

Reframing Dense Action Detection (RefDense): A Paradigm Shift in Problem Solving & a Novel Optimization Strategy

  • 拆分动作类别与运动为独立子任务,由不同网络专注处理。
  • 在Charades和MultiTHUMOS上分别提升3.8%和1.7%的平均性能。
  • 用语言引导对比损失显式建模共现动作关系,适合视频理解研究者。

密集动作检测需识别多个共现且类别模糊的动作,因时间与类别重叠问题复杂,单个网络难以有效应对。本文提出将密集模糊动作检测分解为识别无歧义的子概念(如动作实体与动作运动),并分配给独立子网络,使各网络专注解决时序重叠问题。同时,视频中动作常存在关联性,但现有方法依赖二元交叉熵损失,无法有效学习此类关系。为此,本文引入一种语言引导的对比学习损失,在优化过程中对共现概念提供显式监督。大量实验表明,该方法在挑战性数据集Charades和MultiTHUMOS上显著优于现有最优方法,各项指标平均提升3.8%和1.7%。

原文摘要 · Abstract (English)

Dense action detection involves detecting multiple co-occurring actions while action classes are often ambiguous and represent overlapping concepts. We argue that handling the dual challenge of temporal and class overlaps is too complex to effectively be tackled by a single network. To address this, we propose to decompose the task of detecting dense ambiguous actions into detecting dense, unambiguous sub-concepts that form the action classes (i.e., action entities and action motions), and assigning these sub-tasks to distinct sub-networks. By isolating these unambiguous concepts, the sub-networks can focus exclusively on resolving a single challenge, dense temporal overlaps. Furthermore, simultaneous actions in a video often exhibit interrelationships, and exploiting these relationships can improve the method performance. However, current dense action detection networks fail to effectively learn these relationships due to their reliance on binary cross-entropy optimization, which treats each class independently. To address this limitation, we propose providing explicit supervision on co-occurring concepts during network optimization through a novel language-guided contrastive learning loss. Our extensive experiments demonstrate the superiority of our approach over state-of-the-art methods, achieving substantial improvements of 3.8% and 1.7% on average across all metrics on the challenging benchmark datasets, Charades and MultiTHUMOS.

动作检测视频理解对比学习多任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。