用对抗精炼与密集运动分词，压缩人体动作热图并保留精细轨迹。
Adversarially-Refined VQ-GAN with Dense Motion Tokenization for Spatio-Temporal Heatmaps
- 引入对抗精炼与密集运动分词，提升动作热图压缩质量。
- 在CMU数据集上比dVAE提升9.31% SSIM，时间不稳定性降低37.1%。
- 发现2D动作可用128个词元表示，3D需1024个词元以保真重建。
连续人体动作理解在计算机视觉中仍具挑战，因其高维性与固有冗余。高效压缩与表征对分析复杂运动动态至关重要。本文提出一种对抗精炼的VQ-GAN框架，结合密集运动分词，用于压缩时空热图，同时保留人体动作的细粒度轨迹。该方法消除非对抗基线中的重建伪影（如运动模糊、时间错位）。在CMU Panoptic数据集上的实验表明，本方法相比dVAE基线提升9.31% SSIM，时间不稳定性降低37.1%。此外，密集分词策略揭示：2D动作可由128个词元最优表示，而3D动作复杂度更高，需1024个词元代码本才能实现忠实重建。这些结果为多样动作分析应用提供了实际部署可行性。代码已开源：https://github.com/TeCSAR-UNCC/Pose-Quantization。
原文摘要 · Abstract (English)
Continuous human motion understanding remains a core challenge in computer vision due to its high dimensionality and inherent redundancy. Efficient compression and representation are crucial for analyzing complex motion dynamics. In this work, we introduce an adversarially-refined VQ-GAN framework with dense motion tokenization for compressing spatio-temporal heatmaps while preserving the fine-grained traces of human motion. Our approach combines dense motion tokenization with adversarial refinement, which eliminates reconstruction artifacts like motion smearing and temporal misalignment observed in non-adversarial baselines. Our experiments on the CMU Panoptic dataset provide conclusive evidence of our method's superiority, outperforming the dVAE baseline by 9.31% SSIM and reducing temporal instability by 37.1%. Furthermore, our dense tokenization strategy enables a novel analysis of motion complexity, revealing that 2D motion can be optimally represented with a compact 128-token vocabulary, while 3D motion's complexity demands a much larger 1024-token codebook for faithful reconstruction. These results establish practical deployment feasibility across diverse motion analysis applications. The code base for this work is available at https://github.com/TeCSAR-UNCC/Pose-Quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。