从单目视频中精准还原人体生物力学级3D姿态,无需昂贵设备。
BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videos
- 用多查询变换器提取图像细粒度特征,恢复高精度人体网格。
- 引入神经逆运动学模型,在解剖约束下回归真实关节位置。
- 结合2D姿态信息优化推理过程,适合医疗与机器人场景。
单目视频中的人体3D姿态估计近年依赖参数化模型(如SMPL),但这类模型简化了解剖结构,难以准确捕捉真实关节位置与运动,限制了其在生物力学、医疗和机器人领域的应用。传统生物力学精确的姿态估计需依赖昂贵的标记式动作捕捉系统和实验室优化技术。为填补这一空白,我们提出BioPose,一种基于学习的框架,可直接从单目视频中预测生物力学精确的3D人体姿态。BioPose包含三个核心组件:多查询人体网格恢复模型(MQ-HMR)、神经逆运动学模型(NeurIK)和2D引导的姿态精炼技术。MQ-HMR利用多查询可变形变换器提取多尺度细粒度图像特征,实现精确的人体网格重建。NeurIK将网格顶点视为虚拟标记,通过时空网络在解剖约束下回归生物力学合理的3D姿态。为进一步提升3D姿态估计效果,2D引导精炼步骤在推理阶段通过对齐3D结构与2D姿态观测来优化查询令牌。在基准数据集上的实验表明,BioPose显著优于现有最先进方法。
原文摘要 · Abstract (English)
Recent advancements in 3D human pose estimation from single-camera images and videos have relied on parametric models, like SMPL. However, these models oversimplify anatomical structures, limiting their accuracy in capturing true joint locations and movements, which reduces their applicability in biomechanics, healthcare, and robotics. Biomechanically accurate pose estimation, on the other hand, typically requires costly marker-based motion capture systems and optimization techniques in specialized labs. To bridge this gap, we propose BioPose, a novel learning-based framework for predicting biomechanically accurate 3D human pose directly from monocular videos. BioPose includes three key components: a Multi-Query Human Mesh Recovery model (MQ-HMR), a Neural Inverse Kinematics (NeurIK) model, and a 2D-informed pose refinement technique. MQ-HMR leverages a multi-query deformable transformer to extract multi-scale fine-grained image features, enabling precise human mesh recovery. NeurIK treats the mesh vertices as virtual markers, applying a spatial-temporal network to regress biomechanically accurate 3D poses under anatomical constraints. To further improve 3D pose estimations, a 2D-informed refinement step optimizes the query tokens during inference by aligning the 3D structure with 2D pose observations. Experiments on benchmark datasets demonstrate that BioPose significantly outperforms state-of-the-art methods. Project website: \url{https://m-usamasaleem.github.io/publication/BioPose/BioPose.html}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。