用自适应融合选数据,让少样本模仿学习更准。
COLLAGE: Adaptive Fusion-based Retrieval for Augmented Policy Learning
- 根据任务需求动态融合多类特征,智能筛选相关演示数据
- 仿真环境下比顶尖方法高5.1%,真实场景提升16.6%
- 不依赖特定特征,适合各类少样本学习任务
本文研究少样本模仿学习中的数据检索问题:在仅有少数目标示范的情况下,从大规模数据集中选取有效数据以训练高性能策略。现有方法仅依赖单一特征距离(如视觉、语义或运动空间),易引入无关数据(如因场景布局相似而误选异类任务数据)。为此,提出COLLAGE——一种基于自适应晚期融合的数据聚合方法,通过评估各预选数据子集上训练的策略对目标示范的预测表现,动态分配权重,并据此进行重要性采样。该方法通用且特征无偏,可融合任意数量由不同检索启发式选出的子集,识别出对目标任务最有益的组合。在10个仿真任务和6个真实任务上的实验表明,相较于先进检索与多任务学习方法,分别提升5.1%和16.6%,数据源为大规模DROID数据集。
原文摘要 · Abstract (English)
In this work, we study the problem of data retrieval for few-shot imitation learning: selecting data from a large dataset to train a performant policy for a specific task, given only a few target demonstrations. Prior methods retrieve data using a single-feature distance heuristic, assuming that the best demonstrations are those that most closely resemble the target examples in visual, semantic, or motion space. However, this approach captures only a subset of the relevant information and can introduce detrimental demonstrations, e.g., retrieving data from unrelated tasks due to similar scene layouts, or selecting similar motions from tasks with divergent goals. We present COLLAGE, a method for COLLective data AGgrEgation in few-shot imitation learning that uses an adaptive late fusion mechanism to guide the selection of relevant demonstrations based on a task-specific combination of multiple cues. COLLAGE follows a simple, flexible, and efficient recipe: it assigns weights to subsets of the dataset that are pre-selected using a single feature (e.g., appearance, shape, or language similarity), based on how well a policy trained on each subset predicts actions in the target demonstrations. These weights are then used to perform importance sampling during policy training, sampling data more densely or sparsely according to estimated relevance. COLLAGE is general and feature-agnostic, allowing it to combine any number of subsets selected by any retrieval heuristic, and to identify which subsets provide the greatest benefit for the target task. In extensive experiments, COLLAGE outperforms state-of-the-art retrieval and multi-task learning approaches by 5.1% in simulation across 10 tasks, and by 16.6% in the real world across 6 tasks, where we perform retrieval from the large-scale DROID dataset. More information at https://robin-lab.cs.utexas.edu/COLLAGE .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。