arXiv:2606.11450cs.CV2026-06中稿 · CVPR被引 1

提出自适应掩码重建方法,加速训练并提升动作识别精度。

Exploring Adaptive Masked Reconstruction for Self-Supervised Skeleton-Based Action Recognition

论文配图:Exploring Adaptive Masked Reconstruction for Self-Supervised Skeleton-Based Action Recognition
图 1 · 摘自论文原文
  • 解耦编码器与解码器,灵活预测大时空块,降低重建复杂度。
  • 在三个数据集上训练速度更快,下游识别准确率超越现有方法。
  • 通过动态引导关注关键运动区域,增强模型对动作语义的捕捉能力。

最近,掩码骨骼重建模型成为自监督骨骼动作识别中强大的表示学习方法,推动了该领域显著进展。然而,现有最先进方法需预测大量时空补丁,显著延长训练时间;且在重建过程中同等对待所有时空区域,导致模型被干扰,难以学习关键动作模式。为此,我们提出自适应掩码重建(AMR)框架,实现更快更强的预训练。首先将解码器与编码器解耦,支持灵活预测更大时空块,大幅降低重建复杂度。由于大块包含更复杂信息,难以预测且会降低性能,因此引入自适应引导模块,识别高运动信息量区域,指导模型聚焦每个补丁中最具判别性的部分,缓解重建难度。在NTU RGB+D 60、NTU RGB+D 120和PKU-MMD数据集上的实验表明,AMR不仅显著加快预训练过程,还提升了下游识别准确率,优于当前最先进方法。

原文摘要 · Abstract (English)

Recently, masked skeleton reconstruction models have emerged as strong action representation learners, driving significant progress in self-supervised skeleton-based action recognition. However, existing state-of-the-art methods must predict an exceedingly large number of spatiotemporal patches, significantly prolonging training time. Besides, by treating all spatiotemporal regions equally during reconstruction, these models are distracted from learning the critical motion patterns that underlie action semantics. To address these challenges, we propose Adaptive Masked Reconstruction (AMR), a faster and stronger pre-training framework. We first decouple the decoder from the encoder, enabling flexible prediction of larger spatiotemporal patches and dramatically reducing reconstruction complexity. Given that larger patches contain more complex information, which is challenging to predict and consequently degrades performance, we accordingly introduce an adaptive guidance module. This module identifies regions of high motion informativeness, guiding the model to focus on the most discriminative parts of each patch and alleviating reconstruction difficulty. Experiments on NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD datasets demonstrate that AMR not only accelerates pre-training substantially but also improves downstream recognition accuracy, surpassing current state-of-the-art approaches.

自监督动作识别骨骼建模掩码重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。