arXiv:2509.23455cs.CVcs.LG2025-09

将单目摄像头的3D姿态转为统一朝向,让动作分析不受视角影响。

3DPCNet: Pose Canonicalization for Robust Viewpoint-Invariant 3D Kinematic Analysis from Monocular RGB cameras

  • 用图卷积与注意力融合骨骼局部和全局特征,生成旋转参数对齐姿态
  • 在MM-Fi数据集上将旋转误差从20°降至3.4°,位置误差从64mm降到47mm
  • 适合运动科学、康复评估等需消除视角偏差的动作分析场景

单目3D姿态估计算法输出的是以相机为中心的骨骼坐标,导致运动分析受视角影响。本文提出3DPCNet,一种轻量级、不依赖特定估计算法的模块,可直接处理3D关节坐标,将其校正为一致的以身体为中心的规范坐标系。其混合编码器通过门控交叉注意力机制,融合图卷积网络提取的局部骨骼特征与变压器提供的全局上下文信息。基于该表示,模型预测连续6维旋转,并映射为SO(3)矩阵实现姿态对齐。模型在MM-Fi数据集上通过合成旋转姿势进行自监督训练,采用复合损失函数保证旋转精度与姿态重建质量。在MM-Fi基准测试中,3DPCNet将平均旋转误差从超过20°降低至3.4°,均每关节位置误差从约64mm降至47mm,优于几何基线方法。在TotalCapture数据集上的定性评估显示,本方法从视频生成的加速度信号与真实惯性测量单元(IMU)数据具有强视觉一致性,验证了该模块有效消除了视角差异,实现了物理上合理的运动分析。

原文摘要 · Abstract (English)

Monocular 3D pose estimators produce camera-centered skeletons, creating view-dependent kinematic signals that complicate comparative analysis in applications such as health and sports science. We present 3DPCNet, a compact, estimator-agnostic module that operates directly on 3D joint coordinates to rectify any input pose into a consistent, body-centered canonical frame. Its hybrid encoder fuses local skeletal features from a graph convolutional network with global context from a transformer via a gated cross-attention mechanism. From this representation, the model predicts a continuous 6D rotation that is mapped to an $SO(3)$ matrix to align the pose. We train the model in a self-supervised manner on the MM-Fi dataset using synthetically rotated poses, guided by a composite loss ensuring both accurate rotation and pose reconstruction. On the MM-Fi benchmark, 3DPCNet reduces the mean rotation error from over 20$^{\circ}$ to 3.4$^{\circ}$ and the Mean Per Joint Position Error from ~64 mm to 47 mm compared to a geometric baseline. Qualitative evaluations on the TotalCapture dataset further demonstrate that our method produces acceleration signals from video that show strong visual correspondence to ground-truth IMU sensor data, confirming that our module removes viewpoint variability to enable physically plausible motion analysis.

3D姿态估计视角不变运动分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。