用最优传输优化轨迹匹配，实现无需预训练的零样本模仿学习
Zero-Shot Offline Imitation Learning via Optimal Transport
- 通过最优传输构建状态分布距离，直接优化占用匹配目标
- 在连续复杂环境上实现零样本、非短视模仿，仅需单个示范
- 可从离线次优数据中学习，适合真实场景下的少样本行为复现
零样本模仿学习有望仅凭测试时一个示范就复现未见过的行为。现有方法将专家示范视为一系列目标，依赖高层目标选择器与低层条件策略实现模仿，但易产生短视行为：为达成局部目标而损害长期目标。本文提出新方法，通过最优传输直接优化模仿学习固有的占用匹配目标。我们设计了一种将条件价值函数提升为状态占用间距离的方法，并利用学习的世界模型近似该距离。所提方法可从离线、次优数据中学习，能在复杂连续环境中实现非短视的零样本模仿。代码已开源：https://github.com/martius-lab/zilot。
原文摘要 · Abstract (English)
Zero-shot imitation learning algorithms hold the promise of reproducing unseen behavior from as little as a single demonstration at test time. Existing practical approaches view the expert demonstration as a sequence of goals, enabling imitation with a high-level goal selector, and a low-level goal-conditioned policy. However, this framework can suffer from myopic behavior: the agent's immediate actions towards achieving individual goals may undermine long-term objectives. We introduce a novel method that mitigates this issue by directly optimizing the occupancy matching objective that is intrinsic to imitation learning. We propose to lift a goal-conditioned value function to a distance between occupancies, which are in turn approximated via a learned world model. The resulting method can learn from offline, suboptimal data, and is capable of non-myopic, zero-shot imitation, as we demonstrate in complex, continuous benchmarks. The code is available at https://github.com/martius-lab/zilot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。