通过动作离散化缓解3D人体姿态估计的误差累积问题。
Robust Long-term Test-Time Adaptation for 3D Human Pose Estimation through Motion Discretization
- 用隐空间动作聚类生成锚点动作,提升自监督稳定性。
- 连续适应中误差不增长,长期性能优于已有方法。
- 适合长期视频流中个人姿态建模,如健身追踪场景。
在线测试时自适应通过在无标签测试流上调整模型来缓解训练-测试域差异。然而,3D人体姿态估计的在线自适应因依赖不完美预测的自监督而产生误差累积,导致性能随时间下降。为此,我们提出一种新方案,强调动作离散化:在隐空间中对运动表示进行无监督聚类,得到一组锚点动作,其规律性可辅助姿态估计器并实现高效自回放;同时引入软重置机制,将估计器恢复至指数移动平均状态,以应对持续适应。我们在同一人连续出域视频流上评估长期在线自适应,有效捕捉稳定的个体形体与运动特征。该方法抑制了误差累积,使模型能稳健利用这些个性化特征,提升精度。实验表明,本方案优于现有在线测试时自适应方法,并验证了设计合理性。
原文摘要 · Abstract (English)
Online test-time adaptation addresses the train-test domain gap by adapting the model on unlabeled streaming test inputs before making the final prediction. However, online adaptation for 3D human pose estimation suffers from error accumulation when relying on self-supervision with imperfect predictions, leading to degraded performance over time. To mitigate this fundamental challenge, we propose a novel solution that highlights the use of motion discretization. Specifically, we employ unsupervised clustering in the latent motion representation space to derive a set of anchor motions, whose regularity aids in supervising the human pose estimator and enables efficient self-replay. Additionally, we introduce an effective and efficient soft-reset mechanism by reverting the pose estimator to its exponential moving average during continuous adaptation. We examine long-term online adaptation by continuously adapting to out-of-domain streaming test videos of the same individual, which allows for the capture of consistent personal shape and motion traits throughout the streaming observation. By mitigating error accumulation, our solution enables robust exploitation of these personal traits for enhanced accuracy. Experiments demonstrate that our solution outperforms previous online test-time adaptation methods and validate our design choices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。