端到端捕捉任意骨骼动作，旋转误差降低40%以上。
MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons
- 全端到端设计，联合优化姿态与旋转预测
- 旋转误差从17°降至6.54°，未见骨架也表现稳定
- 无需网格中间态，推理速度提升20倍
从单目视频捕捉任意骨骼动作的现有方法采用分步流程：视频到姿态网络预测关节点位置，再通过非可微的解析逆运动学（IK）恢复旋转。该设计存在固有局限：关节位置无法唯一确定旋转，骨轴扭转等自由度仍模糊，且不可微的IK阶段阻碍系统适应噪声或优化最终动画目标。本文提出首个完全端到端框架，使视频到姿态与姿态到旋转两阶段均可学习并联合优化。我们发现姿态到旋转映射的歧义源于坐标系信息缺失：相同关节点位置在不同静止姿态和局部轴约定下对应不同旋转。为此，引入目标资产的参考姿态-旋转对，结合静止姿态，既锚定映射关系，又定义旋转坐标系。此设定将旋转预测转为良好约束的条件问题，支持有效学习。此外，模型直接从视频预测关节位置，不依赖网格中间表示，提升鲁棒性与效率。两阶段共享骨架感知的全局-局部图引导多头注意力（GL-GMHA）模块，实现关节点级局部推理与全局协调。在Truebones Zoo和Objaverse数据集上的实验表明，本方法将旋转误差从约17度降至约10度，未见骨架上更达6.54度，同时推理速度比基于网格的流水线快约20倍。
原文摘要 · Abstract (English)
Recent methods for arbitrary-skeleton motion capture from monocular video follow a factorized pipeline, where a Video-to-Pose network predicts joint positions and an analytical inverse-kinematics (IK) stage recovers joint rotations. While effective, this design is inherently limited, since joint positions do not fully determine rotations and leave degrees of freedom such as bone-axis twist ambiguous, and the non-differentiable IK stage prevents the system from adapting to noisy predictions or optimizing for the final animation objective. In this work, we present the first fully end-to-end framework in which both Video-to-Pose and Pose-to-Rotation are learnable and jointly optimized. We observe that the ambiguity in pose-to-rotation mapping arises from missing coordinate system information: the same joint positions can correspond to different rotations under different rest poses and local axis conventions. To resolve this, we introduce a reference pose-rotation pair from the target asset, which, together with the rest pose, not only anchors the mapping but also defines the underlying rotation coordinate system. This formulation turns rotation prediction into a well-constrained conditional problem and enables effective learning. In addition, our model predicts joint positions directly from video without relying on mesh intermediates, improving both robustness and efficiency. Both stages share a skeleton-aware Global-Local Graph-guided Multi-Head Attention (GL-GMHA) module for joint-level local reasoning and global coordination. Experiments on Truebones Zoo and Objaverse show that our method reduces rotation error from ~17 degrees to ~10 degrees, and to 6.54 degrees on unseen skeletons, while achieving ~20x faster inference than mesh-based pipelines. Project page: https://animotionlab.github.io/MoCapAnythingV2/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。