将手语生成结果转化为机器人可执行动作，解决肢体穿插问题。
From Sign Language Generation to Humanoid Execution: Vision-Language Guided Retargeting with Collision Mitigation
- 用体积化SMPL-X模型检测并修正肢体穿插，保持原轨迹最小偏移。
- 通过视觉语言模型识别机器人执行中的失败模式并动态修正。
- 适合研究人形机器人手语执行与跨模态运动转换的团队。
近年来的手语生成（SLG）系统输出越来越密集的3D身体表征，更完整地保留全身运动学与几何信息，便于在人形机器人上实现。然而，这些生成动作常出现自交现象，如手手、手躯穿插。此类问题在离线渲染中可容忍，但在机器人执行中会导致逆运动学求解不可行、碰撞及轨迹不稳定。本文提出一个系统级框架，通过两个模块实现从手语生成到人形机器人关节空间执行的桥接：首先引入基于体积化SMPL-X的碰撞缓解模块，将生成动作投影至物理合理配置，同时最小化对原轨迹的偏离；其次提出一种基于逆运动学骨干的视觉-语言引导重定向算法：视觉语言模型作为渲染后人形动作的视觉评判者，识别特定于机器人实现的失败模式，并触发任务空间的针对性修正。实验表明，碰撞处理与感知引导优化是实现可靠人形手语执行的关键缺失组件。
原文摘要 · Abstract (English)
Recent sign language generation (SLG) systems increasingly output dense 3D body representations, which better preserve full-body kinematics and geometry for downstream embodiment on humanoid robots. However, these generated motions frequently exhibit self-intersections such as hand-hand and hand-torso penetration. While such artifacts may be tolerated in offline rendering, they become critical in humanoid execution as they lead to infeasible inverse-kinematics (IK) solutions, collisions, and unstable retargeted trajectories. We present a system-level framework that bridges SLG outputs to humanoid joint-space execution via two components. First, we introduce a volumetric SMPL-X collision-mitigation module that projects generated signing motions toward physically plausible configurations while minimally deviating from the original trajectory. Second, we propose a vision-language-guided retargeting algorithm built on an IK backbone: a VLM serves as a visual critic over rendered humanoid motion, identifies embodiment-specific failure modes, and triggers targeted task-space corrections. Our results highlight collision handling and perception-guided refinement as key missing components for reliable humanoid signing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。