ReMP通过时间注意力机制，实现多模态3D人体姿态的精准追踪与补帧。
ReMP: Reusable Motion Prior for Multi-domain 3D Human Pose Estimation and Motion Inbetweening
- 基于人体参数化模型学习时空运动先验，支持多模态输入
- 在遮挡和噪声下仍能准确估计缺失帧姿态，提升重建效率
- 适用于深度点云、激光雷达、IMU等实际场景，适合动作捕捉应用
我们提出可复用的运动先验（ReMP），一种能精确捕捉多种下游任务中运动时序演化的有效运动先验。受基础模型成功的启发，我们认为一个鲁棒的时空运动先验可封装适用于多种传感器模态的底层3D动态。我们从一系列完整的人体姿态参数化模型序列中学习丰富的运动先验。该先验通过时间注意力机制，在存在显著遮挡的情况下，仍可准确估计缺失帧或噪声测量中的姿态。更有趣的是，该先验能引导系统从不完整且困难的输入中快速提取关键信息，显著提升网格序列恢复的训练效率。ReMP在多样且实用的3D运动数据上持续优于基线方法，涵盖深度点云、激光雷达扫描和惯性测量单元（IMU）数据。项目页面见https://hojunjang17.github.io/ReMP。
原文摘要 · Abstract (English)
We present Reusable Motion prior (ReMP), an effective motion prior that can accurately track the temporal evolution of motion in various downstream tasks. Inspired by the success of foundation models, we argue that a robust spatio-temporal motion prior can encapsulate underlying 3D dynamics applicable to various sensor modalities. We learn the rich motion prior from a sequence of complete parametric models of posed human body shape. Our prior can easily estimate poses in missing frames or noisy measurements despite significant occlusion by employing a temporal attention mechanism. More interestingly, our prior can guide the system with incomplete and challenging input measurements to quickly extract critical information to estimate the sequence of poses, significantly improving the training efficiency for mesh sequence recovery. ReMP consistently outperforms the baseline method on diverse and practical 3D motion data, including depth point clouds, LiDAR scans, and IMU sensor data. Project page is available in https://hojunjang17.github.io/ReMP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。