让机器人通过触觉、语音和图形界面无缝学习与调整技能。
MOMO: A framework for seamless physical, verbal, and graphical robot skill learning and adaptation
- 融合触觉、语音和图形界面实现多模态交互控制
- 支持语音指令完成表面抛光等复杂任务
- 适合非专家在工业场景快速适配机器人
工业机器人应用需要越来越灵活的系统，使非专家用户能够轻松适应不同任务与环境。然而，不同调整方式受益于不同交互模态。我们提出一个交互式框架，通过三种互补模态实现机器人技能自适应：力觉触控用于精确空间修正，自然语言用于高层语义修改，图形化网页界面用于可视化几何关系与轨迹、检查与调整参数，以及通过拖拽编辑途经点。该框架集成五个组件：基于能量的人类意图检测、基于工具的LLM架构（LLM选择并参数化预定义函数而非生成代码）实现安全自然语言适配、核化运动基元（KMPs）用于运动编码、概率虚拟夹具用于引导示范录制，以及遍历控制用于表面抛光。实验表明，该工具型LLM架构可将技能适配从KMPs扩展至遍历控制，实现语音命令下的表面抛光。在7-自由度力控机器人上于Automatica 2025展会上验证了该方法在工业场景中的实用性。
原文摘要 · Abstract (English)
Industrial robot applications require increasingly flexible systems that non-expert users can easily adapt for varying tasks and environments. However, different adaptations benefit from different interaction modalities. We present an interactive framework that enables robot skill adaptation through three complementary modalities: kinesthetic touch for precise spatial corrections, natural language for high-level semantic modifications, and a graphical web interface for visualizing geometric relations and trajectories, inspecting and adjusting parameters, and editing via-points by drag-and-drop. The framework integrates five components: energy-based human-intention detection, a tool-based LLM architecture (where the LLM selects and parameterizes predefined functions rather than generating code) for safe natural language adaptation, Kernelized Movement Primitives (KMPs) for motion encoding, probabilistic Virtual Fixtures for guided demonstration recording, and ergodic control for surface finishing. We demonstrate that this tool-based LLM architecture generalizes skill adaptation from KMPs to ergodic control, enabling voice-commanded surface finishing. Validation on a 7-DoF torque-controlled robot at the Automatica 2025 trade fair demonstrates the practical applicability of our approach in industrial settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。