让机器人学习动作意图而非照搬轨迹,提升适应力与迁移能力。
Mimic Intent, Not Just Trajectories
- 通过多尺度频域分层编码,分离动作意图与执行细节。
- 实现单次迁移:注入意图令牌即可复用技能,成功率超现有方法。
- 适合需要快速适应新环境的机器人操作任务,如工业装配。
尽管模仿学习在灵巧操作中取得显著进展,但当前先进方法如视觉-语言-动作模型仍难以应对环境变化和技能迁移。本文认为根源在于仅模仿原始轨迹而未理解深层意图。为此提出端到端模仿学习新框架MINT:显式解耦行为意图与执行细节。通过多尺度频域标记化,强制对动作片段进行谱分解,粗粒度令牌捕捉低频全局结构,细粒度令牌编码高频细节。生成策略采用逐级自回归,实现从意图到执行的渐进推理,显著提升学习效率与泛化能力。关键优势在于仅需注入演示中的意图令牌即可实现单次技能迁移。在多个操作基准及真实机器人上的实验表明,该方法达到当前最优成功率,推理效率更高,对扰动更鲁棒,并具备高效单次迁移能力。
原文摘要 · Abstract (English)
While imitation learning (IL) has achieved impressive success in dexterous manipulation through generative modeling and pretraining, state-of-the-art approaches like Vision-Language-Action (VLA) models still struggle with adaptation to environmental changes and skill transfer. We argue this stems from mimicking raw trajectories without understanding the underlying intent. To address this, we propose explicitly disentangling behavior intent from execution details in end-2-end IL: Mimic Intent, Not just Trajectories(MINT). We achieve this via multi-scale frequency-space tokenization, which enforces a spectral decomposition of action chunk representation. We learn action tokens with a multi-scale coarse-to-fine structure, and force the coarsest token to capture low-frequency global structure and finer tokens to encode high-frequency details. This yields an abstract Intent token that facilitates planning and transfer, and multi-scale Execution tokens that enable precise adaptation to environmental dynamics. Building on this hierarchy, our policy generates trajectories through next-scale autoregression, performing progressive intent-to-execution reasoning, thus boosting learning efficiency and generalization. Crucially, this disentanglement enables one-shot transfer of skills, by simply injecting the Intent token from a demonstration into the autoregressive generation process. Experiments on several manipulation benchmarks and on a real robot demonstrate state-of-the-art success rates, superior inference efficiency, robust generalization against disturbances, and effective one-shot transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。