arXiv:2503.15918cs.LGcs.AI2025-03

通过去噪机制增强状态转移稳定性,解决模仿学习中的协变量偏移问题。

Denoising-based Contractive Imitation Learning

  • 用去噪网络修正状态预测,实现局部收缩映射,降低误差传播。
  • 在噪声扰动下,多种任务的成功率显著提升,无需额外专家数据。
  • 方法简单易集成,适合实际应用中缺乏环境动态信息的场景。

模仿学习中的核心挑战是协变量偏移问题。现有缓解方法通常需要额外专家交互、环境动态信息或复杂的对抗训练,实用性受限。本文提出一种简单有效的去噪收缩模仿学习方法(DeCIL),通过引入去噪机制增强状态转移映射的收缩性质。该方法训练两个神经网络:一个动力学模型(f)从当前状态预测下一状态,一个联合状态-动作去噪策略网络(d)通过去噪优化状态预测并输出对应动作。理论分析表明,去噪网络作为局部收缩映射,能减少状态转移中的误差传播,提升稳定性。该方法实现简单,可无缝集成至现有模仿学习框架,无需额外专家数据或复杂训练修改。实验结果表明,该方法在噪声扰动下显著提升了多种模仿学习任务的成功率。

原文摘要 · Abstract (English)

A fundamental challenge in imitation learning is the \emph{covariate shift} problem. Existing methods to mitigate covariate shift often require additional expert interactions, access to environment dynamics, or complex adversarial training, which may not be practical in real-world applications. In this paper, we propose a simple yet effective method (DeCIL) to mitigate covariate shift by incorporating a denoising mechanism that enhances the contraction properties of the state transition mapping. Our approach involves training two neural networks: a dynamics model ( f ) that predicts the next state from the current state, and a joint state-action denoising policy network ( d ) that refines this state prediction via denoising and outputs the corresponding action. We provide theoretical analysis showing that the denoising network acts as a local contraction mapping, reducing the error propagation of the state transition and improving stability. Our method is straightforward to implement and can be easily integrated with existing imitation learning frameworks without requiring additional expert data or complex modifications to the training procedure. Empirical results demonstrate that our approach effectively improves success rate of various imitation learning tasks under noise perturbation.

模仿学习去噪稳定性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。