通过捕捉状态间全局与局部变化,提升世界模型推理质量。
GLAM: Global-Local Variation Awareness in Mamba-based World Model
- 引入双分支Mamba结构,分别感知全局与局部状态变化
- 在Atari 100k上取得优于现有方法的归一化人类评分
- 适合需要高效想象力训练的基于模型强化学习场景
在基于模型的强化学习(MBRL)中,模拟真实交互轨迹可提升采样效率。现有方法多直接使用已知状态序列进行推理,但未能捕捉状态间的细微变化,从而限制了推理质量。受人类从变化中推断趋势的启发,本文提出基于Mamba的世界模型GLAM,通过感知和预测状态间的变化来提升推理能力。GLAM包含两个并行的Mamba模块:GMamba关注输入序列中状态变化的全局模式,并利用这些模式增强未来变化的预测;LMamba则聚焦于相邻状态间的变化,以推理未知信息(如奖励、终止信号、视觉表示)。通过融合两模块优势,GLAM能更准确地捕捉环境变化中的高价值变异,使智能体实现更高效的基于想象的训练。实验表明,该方法在Atari 100k基准上的归一化人类评分显著优于现有方法。
原文摘要 · Abstract (English)
Mimicking the real interaction trajectory in the inference of the world model has been shown to improve the sample efficiency of model-based reinforcement learning (MBRL) algorithms. Many methods directly use known state sequences for reasoning. However, this approach fails to enhance the quality of reasoning by capturing the subtle variation between states. Much like how humans infer trends in event development from this variation, in this work, we introduce Global-Local variation Awareness Mamba-based world model (GLAM) that improves reasoning quality by perceiving and predicting variation between states. GLAM comprises two Mambabased parallel reasoning modules, GMamba and LMamba, which focus on perceiving variation from global and local perspectives, respectively, during the reasoning process. GMamba focuses on identifying patterns of variation between states in the input sequence and leverages these patterns to enhance the prediction of future state variation. LMamba emphasizes reasoning about unknown information, such as rewards, termination signals, and visual representations, by perceiving variation in adjacent states. By integrating the strengths of the two modules, GLAM accounts for highervalue variation in environmental changes, providing the agent with more efficient imagination-based training. We demonstrate that our method outperforms existing methods in normalized human scores on the Atari 100k benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。