arXiv:2412.09202cs.CV2024-12AAAI被引 8

通过分层解耦与精炼,提升视频动作定位的准确率

Temporal Action Localization with Cross Layer Task Decoupling and Refinement

  • 用高层语义与底层边界特征解耦分类与定位任务
  • 在五个数据集上达到当前最优,最高提升2.1%平均精度
  • 适合做视频理解与动作识别的科研人员参考

时间动作定位(TAL)需同时完成动作分类与定位,但两类任务对特征需求常冲突。现有方法虽使用独立头,却共享相同输入特征,影响性能。本文提出跨层任务解耦与精炼(CLTDR)方法:利用视频特征金字塔中高层的语义强特征和低层的边界感知特征,有效分离分类与定位任务;并通过跨层多特征融合,优化并对齐两类结果。此外,设计轻量级门控多粒度(GMG)模块,从瞬时、局部和全局三个时间粒度全面提取与聚合特征。得益于CLTDR与GMG模块,本方法在五个挑战性基准(THUMOS14、MultiTHUMOS、EPIC-KITCHENS-100、ActivityNet-1.3、HACS)上均取得当前最优表现,最高平均精度提升达2.1%。代码与预训练模型已公开于https://github.com/LiQiang0307/CLTDR-GMG。

原文摘要 · Abstract (English)

Temporal action localization (TAL) involves dual tasks to classify and localize actions within untrimmed videos. However, the two tasks often have conflicting requirements for features. Existing methods typically employ separate heads for classification and localization tasks but share the same input feature, leading to suboptimal performance. To address this issue, we propose a novel TAL method with Cross Layer Task Decoupling and Refinement (CLTDR). Based on the feature pyramid of video, CLTDR strategy integrates semantically strong features from higher pyramid layers and detailed boundary-aware boundary features from lower pyramid layers to effectively disentangle the action classification and localization tasks. Moreover, the multiple features from cross layers are also employed to refine and align the disentangled classification and regression results. At last, a lightweight Gated Multi-Granularity (GMG) module is proposed to comprehensively extract and aggregate video features at instant, local, and global temporal granularities. Benefiting from the CLTDR and GMG modules, our method achieves state-of-the-art performance on five challenging benchmarks: THUMOS14, MultiTHUMOS, EPIC-KITCHENS-100, ActivityNet-1.3, and HACS. Our code and pre-trained models are publicly available at: https://github.com/LiQiang0307/CLTDR-GMG.

动作定位视频理解特征解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。