用单目视频实时生成机械臂可执行的关节轨迹，精度高且动作自然。
Real-Time ESFP: Estimating, Smoothing, Filtering, and Pose-Mapping
- 四步端到端流程：估计、平滑、滤波、姿态映射
- 自研时空Transformer使骨骼运动连续且符合解剖结构
- 适合低成本桌面机械臂的实时动作捕捉与控制
本文提出ESFP，一个将单目RGB视频转换为低成本4自由度桌面机械臂可执行关节轨迹的端到端系统。该系统包含四个模块：(1) 估计：ROMP将每一帧提升为24关节3维骨骼；(2) 平滑：提出的HPSTM——一种带自注意力的序列到序列Transformer，结合长时序上下文与可微正向运动学解码器，同时预测关节均值与完整协方差，强制保持骨长恒定与解剖合理性；(3) 滤波：基于HPSTM的不确定性估计对根节点归一化轨迹进行方差加权，抑制残余噪声；(4) 姿态映射：几何重定向层将肩-肘-腕三元组映射至uArm的极坐标工作空间，保持手腕朝向不变。
原文摘要 · Abstract (English)
This paper presents ESFP, an end-to-end pipeline that converts monocular RGB video into executable joint trajectories for a low-cost 4-DoF desktop arm. ESFP comprises four sequential modules. (1) Estimating: ROMP lifts each frame to a 24-joint 3-D skeleton. (2) Smoothing: the proposed HPSTM-a sequence-to-sequence Transformer with self-attention-combines long-range temporal context with a differentiable forward-kinematics decoder, enforcing constant bone lengths and anatomical plausibility while jointly predicting joint means and full covariances. (3) Filtering: root-normalized trajectories are variance-weighted according to HPSTM's uncertainty estimates, suppressing residual noise. (4) Pose-Mapping: a geometric retargeting layer transforms shoulder-elbow-wrist triples into the uArm's polar workspace, preserving wrist orientation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。