用模块化世界模型让一个智能体高效学会多个游戏和任务。
Mixture-of-World Models: Scaling Multi-Task Reinforcement Learning with Modular Latent Dynamics
- 分模块压缩视觉信息，用任务条件专家动态建模不同任务
- 单个模型在26个Atari游戏上达110.4%人类基准分，参数减半
- 适合研究通用智能体与高效多任务学习的开发者
多任务强化学习在视觉领域面临样本效率难题，因任务间观测与动态差异大。基于模型的强化学习虽有望提升效率，但传统整体式架构难以捕捉多样动态，导致重建与预测精度差。本文提出混合世界模型（MoW），结合任务自适应的模块化变分自编码器、任务条件专家与共享骨干的混合Transformer动态模型，以及基于梯度的任务聚类策略，实现高效参数分配。在Atari 100k基准上，单一MoW智能体训练26个游戏，平均人类归一化得分110.4%，接近由26个专用模型组成的STORM（114.2%）表现，但参数量减少50%。在Meta-World上，MoW于30万步内达成74.5%平均成功率，刷新纪录。结果表明，MoW为通用世界模型提供了可扩展且参数高效的基底。
原文摘要 · Abstract (English)
A fundamental challenge in multi-task reinforcement learning (MTRL) is achieving sample efficiency in visual domains where tasks exhibit substantial heterogeneity in both observations and dynamics. Model-based reinforcement learning offers a promising path to improved sample efficiency through world models, but standard monolithic architectures struggle to capture diverse task dynamics, resulting in poor reconstruction and prediction accuracy. We introduce Mixture-of-World Models (MoW), a scalable architecture that combines modular variational autoencoders for task-adaptive visual compression, a hybrid Transformer-based dynamics model with task-conditioned experts and a shared backbone, and a gradient-based task clustering strategy for efficient parameter allocation. On the Atari 100k benchmark, a single MoW agent trained once on 26 Atari games achieves a mean human-normalized score of 110.4%, competitive with the score of 114.2% achieved by STORM, an ensemble of 26 task-specific models, while using 50% fewer parameters. On Meta-World, MoW achieves a 74.5% average success rate within 300 thousand environment steps, establishing a new state of the art. These results demonstrate that MoW provides a scalable and parameter-efficient foundation for generalist world models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。