用掩码潜变量变换器提升世界模型精度与效率，首次在1000万步内超越人类专家。
Accurate and Efficient World Modeling with Masked Latent Transformers
- 基于掩码潜变量的Transformer架构，直接在潜空间生成轨迹。
- 在Crafter基准上1000万步内超越人类表现，解锁全部22项成就。
- 兼顾高精度与训练效率，适合强化学习中的高效世界建模场景。
Dreamer算法通过模拟轨迹训练强大智能体，在多个环境领域取得优异表现。然而，其世界模型潜空间压缩导致关键信息丢失，影响智能体性能。尽管Δ-IRIS和DIAMOND等方法通过像素级训练提升了模型准确性，但牺牲了训练效率，且智能体无法利用世界模型的内部表示。本文提出EMERALD（Efficient MaskEd latent tRAnsformer worLD model），采用空间潜状态结合MaskGIT预测，直接在潜空间生成精准轨迹，显著提升智能体性能。在Crafter基准上，EMERALD首次在1000万环境步内超越人类专家表现，并在评估中至少完成全部22项成就。
原文摘要 · Abstract (English)
The Dreamer algorithm has recently obtained remarkable performance across diverse environment domains by training powerful agents with simulated trajectories. However, the compressed nature of its world model's latent space can result in the loss of crucial information, negatively affecting the agent's performance. Recent approaches, such as $Δ$-IRIS and DIAMOND, address this limitation by training more accurate world models. However, these methods require training agents directly from pixels, which reduces training efficiency and prevents the agent from benefiting from the inner representations learned by the world model. In this work, we propose an alternative approach to world modeling that is both accurate and efficient. We introduce EMERALD (Efficient MaskEd latent tRAnsformer worLD model), a world model using a spatial latent state with MaskGIT predictions to generate accurate trajectories in latent space and improve the agent performance. On the Crafter benchmark, EMERALD achieves new state-of-the-art performance, becoming the first method to surpass human experts performance within 10M environment steps. Our method also succeeds to unlock all 22 Crafter achievements at least once during evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。