让机器人通过语言指令实时调整双手动作,无需预先训练
Adapt as You Say: Online Interactive Bimanual Skill Adaptation via Human Language Feedback
- 先理解任务变化,再用扩散模型调节动作
- 六类任务实测表现优于现有方法,跨平台通用性强
- 非专业用户也能用口语指导机器人,适合家庭助手场景
开发能在人类生活环境中自主运行的通用机器人,需具备持续适应任务变化的能力。然而,在部署阶段对高维协同双臂技能进行零样本在线适应仍是根本挑战。本文提出BiSAIL(基于交互语言的双臂技能自适应)框架,通过交互式语言反馈实现离线学习双臂技能的零样本在线适应。其核心思想是采用分层‘推理-调制’范式:首先从多模态任务变化中推断出泛化适应目标,再通过扩散调制生成符合目标的双臂动作。在六类双臂任务和两个双臂机器人平台上开展的大量真实机器人实验表明,该方法在人机协作适应性、任务泛化能力和跨平台可扩展性方面显著优于现有方法。本工作为可通过自然语言灵活定制的双臂智能助手提供了技术基础。实验视频与代码见 https://rip4kobe.github.io/BiSAIL/。
原文摘要 · Abstract (English)
Developing general-purpose robots capable of autonomously operating in human living environments requires the ability to adapt to continuously evolving task conditions. However, adapting high-dimensional coordinated bimanual skills to novel task variations at deployment remains a fundamental challenge. In this work, we present BiSAIL (Bimanual Skill Adaptation via Interactive Language), a novel framework that enables zero-shot online adaptation of offline-learned bimanual skills through interactive language feedback. The key idea of BiSAIL is to adopt a hierarchical reason-then-modulate paradigm, which first infers generalized adaptation objectives from multimodal task variations, and then adapts bimanual motions via diffusion modulation to achieve the inferred objectives. Extensive real-robot experiments across six bimanual tasks and two dual-arm platforms demonstrate that BiSAIL significantly outperforms existing methods in human-in-the-loop adaptability, task generalization and cross-embodiment scalability. This work enables the development of adaptive bimanual assistants that can be flexibly customized by non-expert users via intuitive verbal corrections. Experimental videos and code are available at https://rip4kobe.github.io/BiSAIL/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。