仅用少量状态数据即可实现专家级模仿，突破传统方法对动作数据的依赖。
Latent Wasserstein Adversarial Imitation Learning
- 通过预训练构建动态感知隐空间，仅用状态信息匹配分布
- 在多个MuJoCo环境中，1~2个专家状态片段即达专家性能
- 相比以往对抗式与基于Wasserstein距离的方法更优，适合数据稀缺场景
模仿学习使智能体通过示范学习专家行为。但传统方法需大量中高质量示范及对应动作，常难以获取。为此，我们提出一种新型对抗式模仿学习框架——潜空间Wasserstein模仿学习（LWAIL），聚焦于仅状态分布匹配。该方法在具备动态感知的隐空间中计算Wasserstein距离，其隐空间通过预训练阶段获得：利用少量随机生成的状态数据训练意图条件价值函数（ICVF），以捕捉状态空间的动态结构。实验表明，该机制提升了策略对状态转移的理解能力，使得仅需1至数个仅含状态的专家轨迹即可实现专家级表现。在多个MuJoCo环境中，本方法优于先前基于Wasserstein和对抗式的模仿学习方法，在各类任务中均取得更优结果。
原文摘要 · Abstract (English)
Imitation Learning (IL) enables agents to mimic expert behavior by learning from demonstrations. However, traditional IL methods require large amounts of medium-to-high-quality demonstrations as well as actions of expert demonstrations, both of which are often unavailable. To reduce this need, we propose Latent Wasserstein Adversarial Imitation Learning (LWAIL), a novel adversarial imitation learning framework that focuses on state-only distribution matching. It benefits from the Wasserstein distance computed in a dynamics-aware latent space. This dynamics-aware latent space differs from prior work and is obtained via a pre-training stage, where we train the Intention Conditioned Value Function (ICVF) to capture a dynamics-aware structure of the state space using a small set of randomly generated state-only data. We show that this enhances the policy's understanding of state transitions, enabling the learning process to use only one or a few state-only expert episodes to achieve expert-level performance. Through experiments on multiple MuJoCo environments, we demonstrate that our method outperforms prior Wasserstein-based IL methods and prior adversarial IL methods, achieving better results across various tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。