提出SF-Head模块，提升骨骼动作识别中模糊动作的区分能力。
Synchronized and Fine-Grained Head for Skeleton-Based Ambiguous Action Recognition
- 设计SSTE与AC-FA双机制，平衡时空特征交互
- 在NTU RGB+D等4个数据集上显著提升模糊动作识别准确率
- 轻量级模块可直接插入现有模型，适合实时动作识别场景
基于图卷积网络（GCNs）的骨骼动作识别已取得显著进展，但对“挥手”与“敬礼”等模糊动作的识别仍是重大挑战。现有方法通常采用GCN与时序卷积网络（TCN）串行组合，导致时空特征提取不平衡，且过度关注局部细节而丢失全局上下文，加剧了模糊动作的区分难度。为此，本文提出一种轻量级即插即用模块SF-Head，置于GCN与TCN之间。SF-Head首先通过空间-时间特征蒸馏（SSTE）结合特征冗余损失（F-RL），实现时空特征的均衡交互；随后进行特征聚合与对齐（AC-FA），引入特征一致性损失（F-CL），使聚合特征与原始时空特征保持一致。在NTU RGB+D 60、NTU RGB+D 120、NW-UCLA和PKU-MMD I四个数据集上的实验表明，该方法在模糊动作识别上取得显著提升。
原文摘要 · Abstract (English)
Skeleton-based action recognition using GCNs has achieved remarkable performance, but recognizing ambiguous actions, such as "waving" and "saluting", remains a significant challenge. Existing methods typically rely on a serial combination of GCNs and TCNs, where spatial and temporal features are extracted independently, leading to an unbalanced spatial-temporal information, which hinders accurate action recognition. Moreover, existing methods for ambiguous actions often overemphasize local details, resulting in the loss of crucial global context, which further complicates the task of differentiating ambiguous actions. To address these challenges, we propose a lightweight plug-and-play module called SF-Head, inserted between GCN and TCN layers. SF-Head first conducts SSTE with a Feature Redundancy Loss (F-RL), ensuring a balanced interaction. It then performs AC-FA, with a Feature Consistency Loss (F-CL), which aligns the aggregated feature with their original spatial-temporal feature. Experimental results on NTU RGB+D 60, NTU RGB+D 120, NW-UCLA and PKU-MMD I datasets demonstrate significant improvements in distinguishing ambiguous actions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。