用邻居相似性改进模仿学习,让机器人更适应新环境。
Difference-Aware Retrieval Policies for Imitation Learning
- 基于最近邻匹配专家数据,动态生成动作而非学全局策略
- 在连续控制和机器人操作中提升15%-46%性能
- 无需额外数据或专家反馈,适合真实场景部署
通过行为克隆的参数化模仿学习在部署时可能因分布外状态导致误差累积。我们证明,通过在推理阶段重用训练数据,采用半参数检索式模仿学习方法可缓解该问题。本文提出差异感知检索策略(DARP),将模仿学习重构为局部邻域结构问题,而非直接的状态-动作映射。DARP不依赖全局策略,而是通过k近邻的专家演示及其动作,结合邻近状态与查询状态间的相对距离向量来预测动作。该方法无需额外假设,不需额外数据采集、在线专家反馈或任务特定知识。在连续控制与机器人操作等多样化领域,以及高维视觉特征等多种表示下,相比标准行为克隆性能提升15%-46%。代码与演示见https://weirdlabuw.github.io/darp-site/。
原文摘要 · Abstract (English)
Parametric imitation learning via behavior cloning can suffer from poor generalization to out-of-distribution states due to compounding errors during deployment. We show that reusing the training data during inference via a semi-parametric retrieval-based imitation learning approach can alleviate this challenge. We present Difference-Aware Retrieval Policies for Imitation Learning (DARP), a semi-parametric retrieval-based imitation learning approach that addresses this limitation by reparameterizing the imitation learning problem in terms of local neighborhood structure rather than direct state-to-action mappings. Instead of learning a global policy, DARP trains a model to predict actions based on $k$-nearest neighbors from expert demonstrations, their corresponding actions, and the relative distance vectors between neighbor states and query states. DARP requires no additional assumptions beyond those made for standard behavior cloning -- it does not require additional data collection, online expert feedback, or task-specific knowledge. We demonstrate consistent performance improvements of 15-46% over standard behavior cloning across diverse domains, including continuous control and robotic manipulation, and across different representations, including high-dimensional visual features. Code and demos are available at https://weirdlabuw.github.io/darp-site/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。