arXiv:2502.07591cs.LGcs.AI2025-02NeurIPS被引 12

提出双思维世界模型,让智能体更可靠地进行长期规划。

DMWM: Dual-Mind World Model with Long-Term Imagination

  • 用双系统模拟人类认知,一个直觉推理,一个逻辑推理
  • 在DMControl任务中实现更高逻辑一致性与数据效率
  • 适合需要长期规划的强化学习场景

世界模型中的想象力对智能体以样本高效方式学习长时程策略至关重要。现有基于循环状态空间模型(RSSM)的世界模型依赖单步统计推断捕捉环境动态,因此因预测误差累积而无法完成长期想象任务。受人类认知双过程理论启发,我们提出一种新型双思维世界模型(DMWM)框架,通过整合逻辑推理实现具有逻辑一致性的想象。DMWM由两个组件构成:基于RSSM的系统1(RSSM-S1)以直觉方式处理状态转移,以及融合逻辑的神经网络系统2(LINN-S2),通过分层深度逻辑推理引导想象过程。系统间反馈机制确保想象过程遵循真实环境的逻辑规则。该框架在需长期规划的DMControl基准任务上进行评估。大量实验结果表明,相较于现有先进世界模型,本框架在逻辑一致性、试错效率、数据效率和长期想象能力方面均有显著提升。

原文摘要 · Abstract (English)

Imagination in world models is crucial for enabling agents to learn long-horizon policy in a sample-efficient manner. Existing recurrent state-space model (RSSM)-based world models depend on single-step statistical inference to capture the environment dynamics, and, hence, they are unable to perform long-term imagination tasks due to the accumulation of prediction errors. Inspired by the dual-process theory of human cognition, we propose a novel dual-mind world model (DMWM) framework that integrates logical reasoning to enable imagination with logical consistency. DMWM is composed of two components: an RSSM-based System 1 (RSSM-S1) component that handles state transitions in an intuitive manner and a logic-integrated neural network-based System 2 (LINN-S2) component that guides the imagination process through hierarchical deep logical reasoning. The inter-system feedback mechanism is designed to ensure that the imagination process follows the logical rules of the real environment. The proposed framework is evaluated on benchmark tasks that require long-term planning from the DMControl suite. Extensive experimental results demonstrate that the proposed framework yields significant improvements in terms of logical coherence, trial efficiency, data efficiency and long-term imagination over the state-of-the-art world models.

世界模型长期规划双系统强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。