arXiv:2607.02796cs.CV2026-07

用生物力学模型优化动作捕捉,让手部精细运动更准更稳。

Biomechanics-aware Multi-view Markerless Motion Capture of Dexterous Hand Movements

  • 端到端优化,把生物力学模型嵌入追踪流程
  • 121组数据全成功解算,两阶段方法15%失败
  • 抗遮挡更强,适合临床与康复研究

无标记动作捕捉(MMC)在人体运动生物力学分析中广泛应用,但对手部复杂运动的适用性仍落后于其他骨骼系统。本研究评估了一种将梯度优化与生物力学模型结合的重建方法,用于追踪自由状态下的精细手部动作。使用自研8相机系统,采集6名参与者完成11项任务的121段视频,涵盖6种手部姿势、5种物体操作及近端上肢关节运动。将该方法与主流两阶段方法(先三维重建再施加生物力学约束)对比,结果表明:本文方法在全部121个视频中均成功求解,而两阶段方法有15%未能收敛;剩余视频中,本文方法生成的手部运动学更符合生物力学规律,且在涉及物体操作时对遮挡更具鲁棒性。端到端方法更有效利用了二维关键点信息,自动实现具有生物力学意义的手部运动追踪,可支持临床评估、康复监测与人类运动控制研究。

原文摘要 · Abstract (English)

Markerless motion capture (MMC) techniques have been widely beneficial in biomechanical analysis of human movement; however, application to complex motions of the hand lags other musculoskeletal systems. The primary goal of this study was to evaluate the performance of a biomechanical reconstruction method that implements a gradient-based optimization approach with a biomechanical model in the loop for tracking dexterous, unconstrained hand movements using MMC. Using a custom, 8-camera setup, we acquired 121 video recordings from 6 participants performing 11 different tasks that spanned 6 hand postures, 5 object manipulation tasks, and involved motion of the proximal upper limb joints. Performance of the proposed MMC pipeline was directly compared to a more commonly adopted two-stage reconstruction method that first triangulates 2D keypoints from computer vision pose estimation algorithms to 3D and then enforces biomechanical constraints by solving a constrained inverse kinematics problem. Relative performance was assessed qualitatively by visual inspection and quantitatively using a computer vision metric. Our method generated solutions for all 121 video recordings; the two-stage method did not converge for 15% of the recordings. Across the remaining videos, our method produced more biomechanically plausible hand kinematics than the two-stage method and was more robust to occlusion effects during tasks that involved objects. The relative robustness of the end-to-end method suggests that it is more effective in utilizing the available 2D digital keypoint information. Automatic and biomechanically meaningful tracking of hand kinematics during dexterous movements has the potential to support clinical evaluation, rehabilitation monitoring, and studies of human motor control.

动作捕捉手部建模生物力学计算机视觉

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。