解决人体骨骼遮挡问题，提升动作识别准确率
Parts-Mamba: Augmenting Joint Context with Part-Level Scanning for Occluded Human Skeleton
- 引入部件级扫描与体部融合模块，增强远距离关节上下文建模
- 在NTU RGB+D数据集上遮挡场景下最高提升12.9%准确率
- 适合处理不完整骨骼数据的动作识别任务
骨架动作识别旨在从人体骨架中识别动作。图卷积网络（GCNs）推动了该任务的重大进展。然而在真实场景中，由于人体部分被遮挡或通信质量差，捕获的骨架常不完整或存在缺帧。现有GCN模型因缺失局部上下文而表现不佳。为此，我们提出Parts-Mamba，一种混合GCN-Mamba模型，用于增强对远距离关节上下文信息的捕捉与保持能力。该模型通过部件级扫描特性有效提取部件特异性信息，并利用部件-身体融合模块保留非邻接关节的上下文。在不同遮挡设置下的NTU RGB+D 60和NTU RGB+D 120数据集上进行评估，准确率最高提升12.9%。
原文摘要 · Abstract (English)
Skeleton action recognition involves recognizing human action from human skeletons. The use of graph convolutional networks (GCNs) has driven major advances in this recognition task. In real-world scenarios, the captured skeletons are not always perfect or complete because of occlusions of parts of the human body or poor communication quality, leading to missing parts in skeletons or videos with missing frames. In the presence of such non-idealities, existing GCN models perform poorly due to missing local context. To address this limitation, we propose Parts-Mamba, a hybrid GCN-Mamba model designed to enhance the ability to capture and maintain contextual information from distant joints. The proposed Parts-Mamba model effectively captures part-specific information through its parts-specific scanning feature and preserves non-neighboring joint context via a parts-body fusion module. Our proposed model is evaluated on the NTU RGB+D 60 and NTU RGB+D 120 datasets under different occlusion settings, achieving up to 12.9% improvement in accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。