让世界模型自己发现并修复预测错误,提升规划可靠性。
World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

- 分解动作条件状态预测为可独立验证的合理性与可达性两个因子。
- 在多个任务中实现2倍采样效率提升,下游策略性能提高超22%。
- 适合需要高鲁棒性的机器人规划与世界模型自优化场景。
通用世界模型有望实现可扩展的策略评估、优化与规划,但达到所需鲁棒性仍具挑战。与侧重最优动作的策略学习不同,世界模型需在大量次优动作空间中保持可靠,而这些动作在带标签的机器人交互数据中往往被低估。为此,我们提出世界动作验证器(WAV),一种使世界模型能自我识别预测误差并自改进的框架。核心思想是将动作条件状态预测分解为两个可独立验证的因素:状态合理性与动作可达性。我们证明,由于存在两种底层不对称性——无动作数据更丰富,且动作相关特征维度更低——验证这两个因素比直接前向预测更可行。通过引入来自视频语料库的多样化子目标生成器和从部分状态特征中推断动作的稀疏逆模型,结合所提子目标、推断动作与前向滚动之间的循环一致性约束,WAV在现有方法常失效的探索不足区域提供了有效的验证机制。在涵盖MiniGrid、RoboMimic和ManiSkill的九个任务上,该方法实现了2倍更高的采样效率,并使下游策略性能提升超过22%。
原文摘要 · Abstract (English)
General-purpose world models promise scalable policy evaluation, optimization, and planning, yet achieving the required level of robustness remains challenging. Unlike policy learning which primarily focuses on optimal actions, a world model needs to be reliable over a vast space of suboptimal actions, which are often underrepresented in action-labeled robot interactions. To address this challenge, we propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve. The key idea is to decompose action-conditioned state prediction into two independently verifiable factors: state plausibility and action reachability. We show that verifying these factors is significantly more tractable than direct forward prediction due to two underlying asymmetries: the broader availability of action-free data and the lower dimensionality of action-relevant features. Leveraging these asymmetries, we augment a world model with (i) a diverse subgoal generator obtained from video corpora and (ii) a sparse inverse model that infers actions from a subset of state features. By enforcing cycle consistency among proposed subgoals, inferred actions, and forward rollouts, WAV provides an effective verification mechanism in under-explored regimes, where existing methods often fail. Across nine tasks spanning MiniGrid, RoboMimic, and ManiSkill, our method achieves 2x higher sample efficiency while improving downstream policy performance by over 22%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。