用高层特征预测替代低层坐标重建，提升骨骼动作自监督学习效率与表现
Towards Efficient General Feature Prediction in Masked Skeleton Modeling
- 以高层特征替代原始关节坐标进行预测，构建更高效的自监督框架
- 训练速度比传统方法快6.2倍，且在多个下游任务中达到顶尖性能
- 适用于需要高效骨骼动作建模的场景，如视频理解与人机交互
基于掩码自编码器（MAE）范式近年来显著推动了基于骨骼的动作自监督识别进展。然而，现有方法大多仅限于重建原始关节坐标或其简单变体，导致计算冗余且语义表征能力有限。为此，我们提出一种新型通用特征预测框架（GFP），用于高效掩码骨骼建模。核心创新在于将传统的低层重建替换为从局部运动模式到全局语义表征的高层特征预测。具体而言，引入一个轻量级目标生成网络，在时空层次上动态生成多样化监督信号，避免依赖预计算的离线特征。框架结合约束优化机制，确保特征多样性并防止模型坍塌。在NTU RGB+D 60、NTU RGB+D 120和PKU-MMD数据集上的实验表明，该方法兼具计算高效性（训练速度达标准方法的6.2倍）与优越的表示质量，在多种下游任务中取得领先性能。
原文摘要 · Abstract (English)
Recent advances in the masked autoencoder (MAE) paradigm have significantly propelled self-supervised skeleton-based action recognition. However, most existing approaches limit reconstruction targets to raw joint coordinates or their simple variants, resulting in computational redundancy and limited semantic representation. To address this, we propose a novel General Feature Prediction framework (GFP) for efficient mask skeleton modeling. Our key innovation is replacing conventional low-level reconstruction with high-level feature prediction that spans from local motion patterns to global semantic representations. Specifically, we introduce a collaborative learning framework where a lightweight target generation network dynamically produces diversified supervision signals across spatial-temporal hierarchies, avoiding reliance on pre-computed offline features. The framework incorporates constrained optimization to ensure feature diversity while preventing model collapse. Experiments on NTU RGB+D 60, NTU RGB+D 120 and PKU-MMD demonstrate the benefits of our approach: Computational efficiency (with 6.2$\times$ faster training than standard masked skeleton modeling methods) and superior representation quality, achieving state-of-the-art performance in various downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。