用身体纠正机器人动作，让语言模型机器人更懂人类意图。
Don't Yell at Your Robot: Physical Correction as the Collaborative Interface for Language Model Powered Robots
- 机器人用语言描述场景，自动执行6自由度动态系统指令。
- 人类在运动中物理干预，实时修正机器人动作方向。
- 纠正后动作可转为自然语言，优化后续对话交互。
我们提出一种新方法，通过物理交互实现对语言模型驱动机器人的实时错误纠正，以增强人机协作。与依赖语音或文本命令的方法不同，该机器人利用大语言模型（LLM），基于场景的自然语言描述，主动执行6自由度线性动态系统（DS）指令。在运动过程中，人类可通过物理方式提供修正，用于重新估计目标意图，该意图同样由线性动态系统参数化。经修正的动态系统可转化为自然语言，作为提示词输入，提升未来与语言模型的交互质量。我们在真实+仿真混合实验中验证了该方法的有效性，证明了物理交互作为语言模型驱动人机界面的新范式。
原文摘要 · Abstract (English)
We present a novel approach for enhancing human-robot collaboration using physical interactions for real-time error correction of large language model (LLM) powered robots. Unlike other methods that rely on verbal or text commands, the robot leverages an LLM to proactively executes 6 DoF linear Dynamical System (DS) commands using a description of the scene in natural language. During motion, a human can provide physical corrections, used to re-estimate the desired intention, also parameterized by linear DS. This corrected DS can be converted to natural language and used as part of the prompt to improve future LLM interactions. We provide proof-of-concept result in a hybrid real+sim experiment, showcasing physical interaction as a new possibility for LLM powered human-robot interface.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。