让模型专注识别视频中特定动作，提升多动作场景下的识别精度
Prompt-guided Disentangled Representation for Action Recognition
- 用动态提示引导图网络分离指定动作
- 在多个数据集上超越现有最佳方法
- 适合需要精准定位动作的视频分析任务
动作识别是视频理解的基础任务。现有方法通常提取统一特征处理视频中的所有动作，难以建模多动作场景下不同物体间的交互关系。为缓解此问题，我们探索将指定动作从复杂场景中解耦，提出一种新框架ProDA（Prompt-guided Disentangled Representation for Action Recognition）。该框架利用时空场景图（SSGs），引入动态提示模块（DPM）引导图解析神经网络（GPNN）生成特定动作表示。此外，设计了适应视频的GPNN，通过动态权重聚合信息。实验表明，在视频动作识别任务中，该方法显著优于当前最优方法。
原文摘要 · Abstract (English)
Action recognition is a fundamental task in video understanding. Existing methods typically extract unified features to process all actions in one video, which makes it challenging to model the interactions between different objects in multi-action scenarios. To alleviate this issue, we explore disentangling any specified actions from complex scenes as an effective solution. In this paper, we propose Prompt-guided Disentangled Representation for Action Recognition (ProDA), a novel framework that disentangles any specified actions from a multi-action scene. ProDA leverages Spatio-temporal Scene Graphs (SSGs) and introduces Dynamic Prompt Module (DPM) to guide a Graph Parsing Neural Network (GPNN) in generating action-specific representations. Furthermore, we design a video-adapted GPNN that aggregates information using dynamic weights. Experiments in video action recognition demonstrate the effectiveness of our approach when compared with the state-of-the-art methods. Our code can be found in https://github.com/iamsnaping/ProDA.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。