动态选择关键节点,提升人体交互识别准确率
Learning Adaptive Node Selection with External Attention for Human Interaction Recognition
- 通过自适应计算节点重要性,动态筛选交互关键个体
- 在NTU RGB+D数据集上达到94.2%准确率,优于现有方法
- 适合需要精准捕捉多人交互的视频理解任务
现有基于图卷积网络的方法常将互动个体视为独立图,忽略其内在依赖。尽管近期方法采用预定义交互邻接矩阵整合参与者,但无法自适应捕捉不同动作下的动态与上下文相关联合交互。本文提出主动节点选择与外部注意力网络(ASEA),无需预设假设即可动态捕捉交互关系。方法使用图卷积网络分别建模每位参与者以捕获个体内部关系,从而获得精细动作表征。引入自适应时间节点幅度计算模块(AT-NAC),结合空间运动幅度与自适应时间加权,估计全局节点活跃度,突出显著运动模式并抑制无关信息。通过可学习阈值(受正则化约束)选择最具信息量的节点用于交互建模。设计外部注意力模块在激活节点上运行,有效建模个体间的动态交互与语义关系。大量实验表明,该方法更有效地且灵活地捕捉交互关系,达到当前最优性能。
原文摘要 · Abstract (English)
Most GCN-based methods model interacting individuals as independent graphs, neglecting their inherent inter-dependencies. Although recent approaches utilize predefined interaction adjacency matrices to integrate participants, these matrices fail to adaptively capture the dynamic and context-specific joint interactions across different actions. In this paper, we propose the Active Node Selection with External Attention Network (ASEA), an innovative approach that dynamically captures interaction relationships without predefined assumptions. Our method models each participant individually using a GCN to capture intra-personal relationships, facilitating a detailed representation of their actions. To identify the most relevant nodes for interaction modeling, we introduce the Adaptive Temporal Node Amplitude Calculation (AT-NAC) module, which estimates global node activity by combining spatial motion magnitude with adaptive temporal weighting, thereby highlighting salient motion patterns while reducing irrelevant or redundant information. A learnable threshold, regularized to prevent extreme variations, is defined to selectively identify the most informative nodes for interaction modeling. To capture interactions, we design the External Attention (EA) module to operate on active nodes, effectively modeling the interaction dynamics and semantic relationships between individuals. Extensive evaluations show that our method captures interaction relationships more effectively and flexibly, achieving state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。