让大象软管机器人听懂自然语言指令,生成逼真动作
Whole-Body Semantic-to-Actuation Grounding of Elephant-Inspired Soft-Trunk Motion via Lightweight Flow Matching

- 用轻量流匹配将语言指令转为可执行的肌肉控制信号
- 准确率从25%提升至77.2%,推理速度更快且动作多样
- 适合需要安全柔性交互的机器人应用,如康复陪伴
在近距离人机交互中,类象鼻连续体机械臂能实现丰富的全身表达,但将开放词汇的语义指令转化为此类机器人动作仍具挑战:末端执行器运动无法完整描述身体形态,而直接的全身指令维度高且难以保持可行性。本文提出一种基于轻量流匹配的全身语义到驱动映射框架,用于仿象鼻软管型人机交互。该框架将多模态大模型输出的响应转换为有界、形态对齐的意图-强度元组,采用紧凑的Catmull-Rom样条参数化肌腱驱动轨迹,并利用修正流生成器采样可行的全身象鼻运动。实验表明,该框架将保留测试集的接地正确率从25.0%提升至77.2%(相比原始密集回归基线)。相较于去噪扩散基线,正确率从71.9%提升至77.2%,推理时间由7.86毫秒降至4.87毫秒,同时保持动作多样性。一项包含100名参与者的实体人机交互研究进一步显示,加入生成的软管运动通道后,整体满意度评分从46%提升至82%(相比仅音频视频基线)。
原文摘要 · Abstract (English)
For close-contact human-robot interaction (HRI), trunk-like continuum manipulators provide a physical channel for diverse whole-body expression, but grounding open-vocabulary responses into such robots is difficult: end-effector motion underspecifies body shape, whereas direct whole-body commands are high-dimensional and hard to keep feasible. We propose a whole-body semantic-to-actuation grounding framework for elephant-inspired soft-trunk HRI based on lightweight flow matching. The framework converts responses from a multimodal large language model into bounded, morphology-aligned intent-intensity tuples, parameterizes tendon-actuation trajectories with compact Catmull-Rom spline controls, and uses a rectified-flow generator to sample feasible whole-body trunk motions. Experiments show that the proposed framework improves held-out grounding correctness from 25.0% to 77.2% over a raw-response dense-regression baseline. Compared with a denoising-diffusion baseline, it improves correctness from 71.9% to 77.2% and reduces inference time from 7.86 ms to 4.87 ms while preserving motion diversity. A 100-participant physical HRI study further shows that adding the generated soft-trunk motion channel increases the positive overall-satisfaction rating from 46% to 82% over the audiovisual-only baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。