用两级级联Transformer提升骨骼动作识别准确率
CascadeFormer: A Family of Two-stage Cascading Transformers for Skeleton-based Human Action Recognition
- 先预训练后级联微调，分两阶段学习骨骼特征
- 在三个数据集上表现优异，达当前最优水平
- 适合做动作识别研究与模型复现的开发者
基于骨骼的人体动作识别利用视频中人体关节点坐标序列来识别动作。由于骨骼数据具有固有的时空结构，图卷积网络（GCNs）一直是该领域的主流架构。然而，近年来变压器模型和掩码预训练框架为表示学习开辟了新途径。本文提出CascadeFormer，一种用于骨骼动作识别的两级级联变压器家族。该框架包括一个掩码预训练阶段，用于学习可迁移的骨骼表示，随后是针对判别性动作分类设计的级联微调阶段。我们在三个基准数据集（Penn Action N-UCLA、NTU RGB+D 60）上评估了CascadeFormer，所有任务均取得有竞争力的性能。为促进可复现性，我们发布了代码和模型检查点。
原文摘要 · Abstract (English)
Skeleton-based human action recognition leverages sequences of human joint coordinates to identify actions performed in videos. Owing to the intrinsic spatiotemporal structure of skeleton data, Graph Convolutional Networks (GCNs) have been the dominant architecture in this field. However, recent advances in transformer models and masked pretraining frameworks open new avenues for representation learning. In this work, we propose CascadeFormer, a family of two-stage cascading transformers for skeleton-based human action recognition. Our framework consists of a masked pretraining stage to learn generalizable skeleton representations, followed by a cascading fine-tuning stage tailored for discriminative action classification. We evaluate CascadeFormer across three benchmark datasets (Penn Action N-UCLA, and NTU RGB+D 60), achieving competitive performance on all tasks. To promote reproducibility, we release our code and model checkpoints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。