用变分自编码器检测异常，提升世界模型在规划中的鲁棒性。
Bounding Distributional Shifts in World Modeling through Novelty Detection
- 用变分自编码器作为新颖性检测器，防止规划时偏离训练数据分布
- 在模拟机器人环境中，相比现有方法提升数据效率
- 适合对世界模型鲁棒性有要求的强化学习与规划任务
近期基于视觉的世界模型研究显示，从预训练图像主干网络中获得的潜在状态动态具有显著潜力。然而，当前多数方法对训练质量敏感，需在训练期间近乎完全覆盖动作与状态空间，以避免推理时发散。为提升基于模型的规划算法对学习到的世界模型质量的鲁棒性，本文提出利用变分自编码器作为新颖性检测器，确保规划过程中提出的动作轨迹不会导致模型偏离训练数据分布。通过一系列在挑战性模拟机器人环境中的实验验证，将该方法集成至模型预测控制策略循环中，扩展了DINO-WM架构。结果表明，所提方法在数据效率方面优于现有最先进方案。
原文摘要 · Abstract (English)
Recent work on visual world models shows significant promise in latent state dynamics obtained from pre-trained image backbones. However, most of the current approaches are sensitive to training quality, requiring near-complete coverage of the action and state space during training to prevent divergence during inference. To make a model-based planning algorithm more robust to the quality of the learned world model, we propose in this work to use a variational autoencoder as a novelty detector to ensure that proposed action trajectories during planning do not cause the learned model to deviate from the training data distribution. To evaluate the effectiveness of this approach, a series of experiments in challenging simulated robot environments was carried out, with the proposed method incorporated into a model-predictive control policy loop extending the DINO-WM architecture. The results clearly show that the proposed method improves over state-of-the-art solutions in terms of data efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。