arXiv:2605.16419cs.CVcs.AI2026-05中稿 · EMBC 2026

无需标定和硬件触发,用双摄像头实现居家自同步关节角度监测。

Agentic Pipeline for Self-Synchronized Multiview Joint Angle Monitoring in Uncalibrated Environments

论文配图:Agentic Pipeline for Self-Synchronized Multiview Joint Angle Monitoring in Uncalibrated Environments
图 1 · 摘自论文原文
  • 用大模型自动对齐多视角视频并验证,实现无标定环境下的自同步。
  • 在未标定环境下关节角估计误差仅5.97°±2.36°,相关性达0.962。
  • 适合脊髓损伤患者在家长期康复使用,支持多人与遮挡场景。

运动学监测对脊髓损伤(SCI)患者的长期康复至关重要,多视角无标记动作捕捉技术展现出巨大潜力。然而,其部署受限于标定依赖和多视角同步难题,难以在患者自设环境中应用。本文提出一种代理式流水线,在无标定、无硬件触发的双摄像头环境下实现自同步多视角关节角度监测。多模态大语言模型用于自动视频同步与代理驱动的自我验证;采用先进单目2D姿态估计模型提取候选姿态,通过代理机制自动识别并跟踪目标主体,有效应对多人及遮挡情况,生成一致的2D姿态序列。基于非标定多视角姿态序列,结合显式几何建模优化关节角度估计,确保结果可解释性。与Vicon系统对比验证显示,该方法平均绝对误差为5.97°±2.36°,皮尔逊相关系数达0.962±0.014。所提方法有望为患者提供实用、可自部署的居家日常运动学监测系统。

原文摘要 · Abstract (English)

Kinematic monitoring plays a critical role in long-term rehabilitation for patients with spinal cord injury (SCI), where multi-view markerless motion capture methods have shown significant potential. However, owing to the reliance on calibration and the difficulty of achieving multi-view synchronization, their deployment in patient self-deployed environments remains challenging. In this work, we propose an agentic pipeline for self-synchronized multi-view joint angle monitoring in uncalibrated environments using two cameras without hardware triggers. The Multimodal large language models enable automatic video synchronization and agent-driven self-verification. State-of-the-art monocular 2D pose estimation models are employed to extract candidate poses, where an agent-based selection mechanism is then applied to automatically identify and track the target subject, thereby producing consistent 2D poses in the presence of multiple individuals and occlusions. Such 2D poses are optimized to estimate joint angles from uncalibrated multi-view pose sequences, ensuring interpretability through explicit geometric modeling. Validation against Vicon system demonstrated the strong performance, achieving an MAE of $5.97^\circ \pm 2.36^\circ$ and a Pearson correlation coefficient of $0.962 \pm 0.014$. The proposed method is expected to provide a practical, patient self-deployable system to perform daily kinematic monitoring in uncalibrated home environments.

运动捕捉康复监测多视角同步大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。