提出MASQ框架,解决骨架动作分割中掩码与量化导致的抖动问题。
MASQ: Mask-Aware Spatiotemporal Quantization for Unsupervised Skeleton Action Segmentation

- 分维度处理:空间用关节轨迹掩码,时间用可见关节速度损失
- 在三个数据集上平均帧精度显著领先现有方法
- 适合做无监督骨架动作分割的研究者和工程师参考
无监督骨架动作分割是理解长视频中人类行为的关键任务。现有方法常依赖离散量化来发现动作边界,但引入空间掩码后会引发表征模糊,离散量化又放大潜在空间微小波动,导致边界处严重时序抖动。为此,我们提出掩码感知时空量化(MASQ)框架,将空间特征推理与时间平滑解耦。空间上采用联合层级结构丢弃(JLSD),对选定关节的完整时序轨迹进行掩码,促进模型学习关节间协调模式;时间上设计掩码感知速度损失,仅对可见关节强制运动一致性,避免被掩码信号引发的梯度冲突,稳定时序预测。在HuGaDB、LARa、BABEL三个常用骨架数据集上的大量实验表明,本方法显著优于现有最先进无监督方法,尤其在平均帧精度(Mean over Frames)上取得全面且显著的优势。
原文摘要 · Abstract (English)
Unsupervised skeleton-based temporal action segmentation is a crucial task for understanding human behavior in long untrimmed sequences. Recent approaches often rely on discrete quantization to discover action boundaries from motion representations. However, when spatial masking is introduced for representation learning, it can introduce representation ambiguity, while discrete quantization further amplifies small fluctuations in the latent space. The interaction between these two factors often leads to unstable code switching and severe temporal jitter near action boundaries.To address these limitations, we propose a novel Mask-aware Action Spatiotemporal Quantization (MASQ) framework. Our framework decouples the conflicting tasks of spatial feature inference and temporal smoothing.In the spatial dimension, we introduce a Joint-Level Structured Dropout (JLSD) mechanism that masks the entire temporal trajectory of selected joints, to encourage the model to learn discriminative inter-joint coordination patterns. In the temporal dimension, we design a mask-aware velocity loss that enforces motion consistency only on visible joints, that prevents gradient conflicts caused by masked signals and stabilizing temporal predictions. Extensive experiments on three widely used skeleton datasets, including HuGaDB, LARa, and BABEL, demonstrate that the proposed MASQ framework significantly outperforms existing state-of-the-art unsupervised methods. In particular, our model establishes a comprehensive and substantial leading advantage in the Mean over Frames accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。