通过全局Lipschitz正则化提升行为克隆的鲁棒性，防止观测误差导致决策失误。
Robust Behavior Cloning Via Global Lipschitz Regularization
- 引入全局Lipschitz正则化，使策略网络对观测扰动更稳定。
- 理论证明该方法可抵御有界范数范围内的扰动，提供可量化鲁棒性保证。
- 适用于自动驾驶等对安全性要求高的场景，适合关注模型鲁棒性的研究者。
行为克隆（BC）是一种有效的模仿学习技术，已被应用于自动驾驶等安全关键领域。它仅使用专家示范的状态-动作配对数据训练策略，无需与环境交互。然而在部署时，观测可能包含测量误差或对抗性干扰，导致观测偏离真实状态，进而误导智能体做出次优动作。本文提出一种全局Lipschitz正则化方法，增强学习策略网络的鲁棒性。我们证明该方法所获得的全局Lipschitz性质，能为策略提供针对不同有界范数扰动的鲁棒性证书。进一步提出构建Lipschitz神经网络的方法，以确保策略鲁棒性。我们在Gymnasium中的多个环境上实证验证了该理论的有效性。
原文摘要 · Abstract (English)
Behavior Cloning (BC) is an effective imitation learning technique and has even been adopted in some safety-critical domains such as autonomous vehicles. BC trains a policy to mimic the behavior of an expert by using a dataset composed of only state-action pairs demonstrated by the expert, without any additional interaction with the environment. However, During deployment, the policy observations may contain measurement errors or adversarial disturbances. Since the observations may deviate from the true states, they can mislead the agent into making sub-optimal actions. In this work, we use a global Lipschitz regularization approach to enhance the robustness of the learned policy network. We then show that the resulting global Lipschitz property provides a robustness certificate to the policy with respect to different bounded norm perturbations. Then, we propose a way to construct a Lipschitz neural network that ensures the policy robustness. We empirically validate our theory across various environments in Gymnasium. Keywords: Robust Reinforcement Learning; Behavior Cloning; Lipschitz Neural Network
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。