arXiv:2608.29928cs.CV2026-08中稿 · ECCV

从单张图片直接预测生物力学关节角,精度接近顶尖方法。

Biomechanical 3D Body: Self-Supervised Distillation of Biomechanical Pose from a 3D Body Foundation Model

论文配图:Biomechanical 3D Body: Self-Supervised Distillation of Biomechanical Pose from a 3D Body Foundation Model
图 1 · 摘自论文原文
  • 用自监督蒸馏法从3D人体基础模型中提取生物力学关节参数。
  • 在MoVi和BioCV数据集上优于现有直接回归方法,仅略逊于最优轨迹优化法。
  • 适合临床与运动分析场景,可高效生成生物力学输出。

当前单目人体重建方法预测网格顶点和骨骼树上的角度,但其输出缺乏下游临床与生物力学分析所需的定义明确的关节角。本文扩展现有基础模型SAM-3D-Body,增加一个生物力学预测头,可从单张RGB图像回归生物力学模型的关节角与尺度。由于配对图像与生物力学拟合数据稀缺,训练面临挑战。为此,采用莱文伯格-马尔夸特求解器对网格预测中的标记点进行逆向运动学拟合,生成闭环优化目标,实现无需标签的自监督蒸馏。整个模型基于JAX与Equinox实现,适配MuJoCo的GPU优化生物力学模型。在公开发布的SAM-3D-Body数据集上训练后,验证于两个公开标记数据集MoVi、BioCV,以及多视角无标记捕捉的临床队列数据。结果表明,该模型在直接从图像回归生物力学方面优于现有方法,仅略逊于需高成本推理时优化轨迹的最先进方法。

原文摘要 · Abstract (English)

State-of-the-art monocular body recovery methods predict mesh vertices and angles on the corresponding kinematic tree, but their outputs lack biomechanically defined joint angles that downstream applications like clinical and biomechanical analyses require. We extend an existing foundation model, SAM-3D-Body, with an additional biomechanical prediction head that, from a single RGB image, regresses the joint angles and scales of a biomechanical model. Training this model presents a challenge, as there are limited datasets of paired images and biomechanical fits. To overcome this, we supervise biomechanical outputs with in-loop optimized targets from a Levenberg-Marquardt solver performing inverse kinematics fits against markers from the mesh predictions. This allows distilling the biomechanical head from the mesh head, even from unlabeled images. To make this work with GPU-optimized biomechanical models in MuJoCo, the entire model was implemented in JAX using Equinox. We trained this distilled output head on the publicly released SAM-3D-Body dataset. We then validated this model on biomechanical fits to two publicly available marker-based datasets, MoVi and BioCV, as well as movements from a clinical cohort captured with multiview markerless motion capture. The resulting model outperforms existing models for direct regression of biomechanics from images while only slightly underperforming the state-of-the-art monocular biomechanics method that performs more costly inference-time optimization of entire trajectories.

生物力学3D人体重建自监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。