用一次示范教机器人新任务，无需重训
Behavior Prompting Policy: Demonstrations as Prompts for Manipulation
- 通过视觉动作架构将示范转化为指令，实时生成动作
- 在未见任务上实现快速适应，新任务成功率超70%
- 适合希望零样本教机器人的研究者与工程师
我们研究行为提示（behavior prompting）范式，即在推理时仅需一个示范（行为提示）即可让机器人执行新任务。为此，我们提出行为提示策略（BPP），一种上下文感知的视觉-运动架构，能将行为提示与当前观测映射为机器人动作。我们发现任务多样性是提示能力的关键驱动因素，并引入iPhUMI——一种手持操作界面，用于收集多样化的训练数据。在评估方面，我们提出DrawAnything和LIBERO-Gen两个基准，用于测试模型对未见绘画与桌面操作任务的推理适应能力。实验表明，iPhUMI可作为测试时的行为提示接口，使人类仅通过一次示范即可指挥机器人完成已知任务或定义新功能。整体上，行为提示提供了一种灵活、可扩展的无须昂贵微调的新技能学习方式。
原文摘要 · Abstract (English)
We study behavior prompting, a paradigm that enables robots to perform new tasks at inference time given a single human demonstration, which we call a behavior prompt. To enable this capability, we present contributions in algorithm, data, and evaluation. For algorithm, we introduce Behavior Prompting Policy (BPP), an in-context visuomotor architecture that translates the behavior prompt and the current observation into robot actions. For data, we identify that task diversity is the primary driver of the prompting capability and introduce iPhUMI, a handheld manipulation interface for collecting diverse training data. For evaluation, we introduce DrawAnything and LIBERO-Gen to evaluate test-time adaptation to unseen drawing and tabletop manipulation tasks. We also demonstrate that iPhUMI serves as a practical interface for specifying behavior prompts at test time, enabling a human to command a robot via a single demonstration to complete known tasks or to define new robot capabilities. Altogether, behavior prompting provides a flexible and scalable way to teach robots new skills without the need for expensive fine-tuning. Our project website is located at https://behavior-prompting.github.io/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。