用自然语言控制机器人完成复杂操作，无需重新训练
STEER: Flexible Robotic Manipulation via Dense Language Grounding
- 通过语言标注训练可组合的模块化操作技能
- 新任务无需额外数据，直接合成新行为
- 适合需要灵活应变的工业与服务机器人场景
现实世界复杂多变，要求机器人能智能适应未见情境。我们提出STEER框架，将高层常识推理与低层精准控制相连接。该方法通过密集语言标注训练语言接地策略，将复杂情境认知转化为可执行的底层行为。基于以自然语言表达的基本、模块化操作技能构建策略训练，使人类或视觉-语言模型可通过任务与上下文推理，智能编排机器人行为。实验表明，通过STEER学习的技能可组合生成新行为，以适应新情境或执行全新任务，且无需额外数据采集或训练。
原文摘要 · Abstract (English)
The complexity of the real world demands robotic systems that can intelligently adapt to unseen situations. We present STEER, a robot learning framework that bridges high-level, commonsense reasoning with precise, flexible low-level control. Our approach translates complex situational awareness into actionable low-level behavior through training language-grounded policies with dense annotation. By structuring policy training around fundamental, modular manipulation skills expressed in natural language, STEER exposes an expressive interface for humans or Vision-Language Models (VLMs) to intelligently orchestrate the robot's behavior by reasoning about the task and context. Our experiments demonstrate the skills learned via STEER can be combined to synthesize novel behaviors to adapt to new situations or perform completely new tasks without additional data collection or training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。