提出新方法让3D人体姿态估计无视拍摄角度变化,适合部署在边缘设备。
MoViD: View-Invariant 3D Human Pose Estimation via Motion-View Disentanglement

- 通过分离视角信息与动作特征,提升模型对不同拍摄角度的鲁棒性
- 在9个数据集上误差降低超24.2%,仅用60%训练数据仍表现稳定
- 支持15帧/秒实时推理,适合无人机、步态分析等边缘场景
3D人体姿态估计是健康监测、人机协作和沉浸式游戏的关键技术,但真实场景中受视角变化影响严重。现有方法难以泛化到未见视角,需大量训练数据且推理延迟高。本文提出MoViD框架,通过解耦视角信息与运动特征实现视角不变的3D姿态估计。核心思路是从中间姿态特征中提取视角信息,并利用其增强估计的鲁棒性与效率。MoViD引入视点估计器建模关键关节关系以预测视角,采用正交投影模块解耦运动与视角特征,并通过跨视角物理引导对比对齐进一步优化。为支持实时边缘部署,采用逐帧推理流程,结合视点感知策略动态激活翻转精修。在九个公开数据集及新采集的多视角无人机与步态分析数据集上评估显示,相比当前最优方法,姿态估计误差降低超过24.2%,在严重遮挡下仅需60%训练数据即可保持鲁棒性能,并在NVIDIA边缘设备上实现15 FPS的实时推理。
原文摘要 · Abstract (English)
3D human pose estimation is a key enabling technology for applications such as healthcare monitoring, human-robot collaboration, and immersive gaming, but real-world deployment remains challenged by viewpoint variations. Existing methods struggle to generalize to unseen camera viewpoints, require large amounts of training data, and suffer from high inference latency. We propose MoViD, a viewpoint-invariant 3D human pose estimation framework that disentangles viewpoint information from motion features. The key idea is to extract viewpoint information from intermediate pose features and leverage it to enhance both the robustness and efficiency of pose estimation. MoViD introduces a view estimator that models key joint relationships to predict viewpoint information, and an orthogonal projection module to disentangle motion and view features, further enhanced through physics-grounded contrastive alignment across views. For real-time edge deployment, MoViD employs a frame-by-frame inference pipeline with a view-aware strategy that adaptively activates flip refinement based on the estimated viewpoint. Evaluations on nine public datasets and newly collected multiview UAV and gait analysis datasets show that MoViD reduces pose estimation error by over 24.2\% compared to state-of-the-art methods, maintains robust performance under severe occlusions with 60\% less training data, and achieves real-time inference at 15 FPS on NVIDIA edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。