通过分解与补全骨架序列,提升时间损坏下细粒度动作识别准确率。
FineTec: Fine-Grained Action Recognition Under Temporal Corruption via Skeleton Decomposition and Sequence Completion
- 用上下文感知补全和多样的时间掩码恢复基础骨架序列。
- 在严重时间损坏下,对Gym99和Gym288分别达到89.1%和78.1%准确率。
- 适合处理在线姿态估计缺失数据的细粒度动作识别场景。
从时间损坏的骨架序列中识别细粒度动作仍是重大挑战,尤其在真实场景中在线姿态估计常导致大量缺失数据。现有方法难以准确恢复时间动态与细微空间结构,丢失区分相似动作的关键运动线索。为此,我们提出FineTec,一个针对时间损坏下的细粒度动作识别统一框架。FineTec首先利用上下文感知补全与多样时间掩码,从损坏输入中恢复基础骨架序列;接着,基于骨骼的空间分解模块将骨架划分为五个语义区域,根据运动方差分为动态与静态子组,并通过定向扰动生成两个增强序列;这些序列连同基础序列,经由基于拉格朗日动力学的物理驱动估计模块,估算关节加速度;最后,融合的位置序列与加速度序列共同输入基于图卷积网络的动作识别头。在粗粒度(NTU-60、NTU-120)与细粒度(Gym99、Gym288)基准上的广泛实验表明,FineTec在多种时间损坏水平下显著优于此前方法。尤其在极端损坏设置下,于Gym99-severe与Gym288-severe上分别取得89.1%与78.1%的顶级分类准确率,证明其鲁棒性与泛化能力。代码与数据集见:https://smartdianlab.github.io/projects-FineTec/
原文摘要 · Abstract (English)
Recognizing fine-grained actions from temporally corrupted skeleton sequences remains a significant challenge, particularly in real-world scenarios where online pose estimation often yields substantial missing data. Existing methods often struggle to accurately recover temporal dynamics and fine-grained spatial structures, resulting in the loss of subtle motion cues crucial for distinguishing similar actions. To address this, we propose FineTec, a unified framework for Fine-grained action recognition under Temporal Corruption. FineTec first restores a base skeleton sequence from corrupted input using context-aware completion with diverse temporal masking. Next, a skeleton-based spatial decomposition module partitions the skeleton into five semantic regions, further divides them into dynamic and static subgroups based on motion variance, and generates two augmented skeleton sequences via targeted perturbation. These, along with the base sequence, are then processed by a physics-driven estimation module, which utilizes Lagrangian dynamics to estimate joint accelerations. Finally, both the fused skeleton position sequence and the fused acceleration sequence are jointly fed into a GCN-based action recognition head. Extensive experiments on both coarse-grained (NTU-60, NTU-120) and fine-grained (Gym99, Gym288) benchmarks show that FineTec significantly outperforms previous methods under various levels of temporal corruption. Specifically, FineTec achieves top-1 accuracies of 89.1% and 78.1% on the challenging Gym99-severe and Gym288-severe settings, respectively, demonstrating its robustness and generalizability. Code and datasets could be found at https://smartdianlab.github.io/projects-FineTec/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。