通过分层技能检索,让机器人用更少演示学会新任务。
Hierarchical Skill Retrieval for Data-Efficient Adaptation of Vision-Language-Action Models

- 将任务拆解为候选技能序列,按语义合理性和可靠性评估
- 结合语言匹配与行为特征重排序,找到最适配的示范
- 适合数据稀缺下的机器人任务快速适应,尤其长时序操作
尽管在大规模机器人数据集上预训练的视觉-语言-动作(VLA)模型为机器人操作提供了强大基础,但在仅有少量特定任务示范的情况下,其性能可能下降。检索是一种高效利用已有示范进行数据高效适配的实用方法,但现有方法多依赖视觉相似性、状态-动作表示或任务级语言匹配,常忽略长时序操作任务中的层级结构——完整任务匹配罕见,但可复用的技能却很丰富。为此,我们提出分层技能检索(HSR)框架,用于数据高效的VLA适配。HSR首先将目标任务分解为候选技能序列,并基于语义合理性与从先验数据集估计的技能可靠性进行评估。选定分解后,采用混合检索:结合子任务级语言检索与行为特征重排序,识别出既语义相关又与目标任务兼容的示范。最后,通过两阶段预训练与微调流程适配策略,分离通用技能学习与任务特定适应。在LIBERO基准和多个真实机器人操作任务上的实验表明,相较于最强基线,HSR分别提升了10.3%和21.3%的平均成功率。结果证明,结构化技能级检索对数据高效的VLA适配有效。视频与代码见 https://hoar012.github.io/HSR-Project。
原文摘要 · Abstract (English)
While Vision-Language-Action (VLA) models pretrained on large-scale robot datasets provide a strong foundation for robot manipulation, their performance can degrade when adapted to new tasks with limited task-specific demonstrations. Retrieval offers a practical way to reuse existing demonstrations for data-efficient adaptation, but existing methods often rely on visual similarity, state-action representations, or task-level language matching. These approaches may overlook the hierarchical structure of long-horizon manipulation tasks, where complete task matches are rare but reusable skills are often abundant. To address this challenge, we propose Hierarchical Skill Retrieval (HSR), a retrieval framework for data-efficient VLA adaptation. Specifically, HSR first decomposes a target task into candidate skill sequences. It evaluates each plan based on both semantic plausibility and skill reliability estimated from the prior dataset. The selected decomposition is then used for hybrid retrieval. This combines subtask-level language retrieval with behavior-feature reranking to identify demonstrations that are both semantically relevant and compatible with the target task. Finally, we adapt the policy through a two-stage pretraining and finetuning pipeline, which separates general skill acquisition from task-specific adaptation. Experiments on the LIBERO benchmark and several real-world robot manipulation tasks show that HSR improves the average success rate by 10.3% and 21.3% over the strongest baseline, respectively. These results demonstrate the effectiveness of structured skill-level retrieval for data-efficient VLA adaptation. Videos and code are available at https://hoar012.github.io/HSR-Project.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。