用不确定性加权提升模型效率，让机器人控制更稳定
WIMLE: Uncertainty-Aware World Models with IMLE for Sample-Efficient Continuous Control
- 基于IMLE构建多模态世界模型，无需迭代采样
- 通过置信度加权合成轨迹，减少错误预测干扰
- 在14个复杂任务中提升50%样本效率，适合高要求控制场景
基于模型的强化学习虽具高样本效率潜力，但常因模型误差累积、单峰模型对多模态动态的平均以及过度自信预测导致性能不佳。本文提出WIMLE，将隐式最大似然估计（IMLE）扩展至基于模型的RL框架，无需迭代采样即可学习随机、多模态的世界模型，并通过集成与隐变量采样估计预测不确定性。训练时，WIMLE根据预测置信度为每个合成轨迹加权，保留有效回放，削弱不确定预测带来的偏差，实现稳定学习。在涵盖DeepMind Control、MyoSuite和HumanoidBench的40个连续控制任务中，WIMLE在样本效率上表现卓越，渐近性能优于或媲美强基线模型。尤其在挑战性的Humanoid-run任务中，相较最强对手提升超过50%的样本效率；在HumanoidBench上成功解决8/14任务（对比BRO为4/14，SimbaV2为5/14）。结果凸显了基于IMLE的多模态建模与不确定性感知加权对稳定模型化强化学习的关键价值。
原文摘要 · Abstract (English)
Model-based reinforcement learning promises strong sample efficiency but often underperforms in practice due to compounding model error, unimodal world models that average over multi-modal dynamics, and overconfident predictions that bias learning. We introduce WIMLE, a model-based method that extends Implicit Maximum Likelihood Estimation (IMLE) to the model-based RL framework to learn stochastic, multi-modal world models without iterative sampling and to estimate predictive uncertainty via ensembles and latent sampling. During training, WIMLE weights each synthetic transition by its predicted confidence, preserving useful model rollouts while attenuating bias from uncertain predictions and enabling stable learning. Across $40$ continuous-control tasks spanning DeepMind Control, MyoSuite, and HumanoidBench, WIMLE achieves superior sample efficiency and competitive or better asymptotic performance than strong model-free and model-based baselines. Notably, on the challenging Humanoid-run task, WIMLE improves sample efficiency by over $50$\% relative to the strongest competitor, and on HumanoidBench it solves $8$ of $14$ tasks (versus $4$ for BRO and $5$ for SimbaV2). These results highlight the value of IMLE-based multi-modality and uncertainty-aware weighting for stable model-based RL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。