arXiv:2604.05320cs.RO2026-04

让机器人在协作中表达意图,还能听懂并响应人类打断。

ExpressMM: Expressive Mobile Manipulation Behaviors in Human-Robot Interactions

  • 用视觉语言模型实现语言引导的高层规划与低层行为生成
  • 支持用户中途打断或修改指令,交互更灵活自然
  • 实测显示观众能清晰理解机器人意图,认为其安全可预测

移动操作机器人在人机协作环境中日益普及。为有效完成任务,它们还需通过富有表现力的行为传达意图。现有方法多依赖预设动作或示范学习,或使用大语言模型生成高层交互,但未充分考虑用户在任务执行中可能中断、修改或重定向机器人行为的情形。本文提出ExpressMM框架,结合基于视觉-语言模型的高层语言引导规划器与低层视觉-语言-动作策略,实现协作式人机交互中的表现性行为生成。该框架支持可中断交互,可实时响应用户的更新或重定向指令。我们在移动操作机器人协助人类完成协同装配任务的场景中验证了ExpressMM,并开展面向观众的现场人机交互演示评估。问卷结果显示,采用ExpressMM的机器人表现出更清晰的动作意图,促进社会恰当且易懂的互动;参与者普遍认为机器人在协作中实用、行为可预测且安全,提升了对机器人有用性、安全性与可预测性的正面评价。

原文摘要 · Abstract (English)

Mobile manipulators are increasingly deployed in human-centered environments to perform tasks. While completing such tasks, they should also be able to communicate their intent to the people around them using expressive robot behaviors. Prior work on expressive robot behaviors has used preprogrammed or learning-from-demonstration-based expressive motions and large language model generated high-level interactions. The majority of these existing approaches have not considered human-robot interactions (HRI) where users may interrupt, modify, or redirect a robot's actions during task execution. In this paper, we develop the novel ExpressMM framework that integrates a high-level language-guided planner based on a vision-language model for perception and conversational reasoning with a low-level vision-language-action policy to generate expressive robot behaviors during collaborative HRI tasks. Furthermore, ExpressMM supports interruptible interactions to accommodate updated or redirecting instructions by users. We demonstrate ExpressMM on a mobile manipulator assisting a human in a collaborative assembly scenario and conduct audience-based evaluation of live HRI demonstrations. Questionnaire results show that the ExpressMM-enabled expressive behaviors helped observers clearly interpret the robot's actions and intentions while supporting socially appropriate and understandable interactions. Participants also reported that the robot was useful for collaborative tasks and behaved in a predictable and safe manner during the demonstrations, fostering positive perceptions of the robot's usefulness, safety, and predictability during the collaborative tasks.

人机交互移动操作表现性行为可中断交互

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。