用随机探索生成自动数据,训练多环境世界模型。
Exploration-Driven Generative Interactive Environments
- 基于世界模型不确定性驱动的探索代理,无需环境奖励
- 在974个虚拟环境中实现快速适应,提升视频保真度与可控性
- 开源数据集RetroAct和代码,适合仿真与自动生成研究者
现代世界模型需要大量耗时的人类或特定环境代理的动作示范视频数据。为简化训练,我们采用大量虚拟环境,通过自动采集交互数据。尽管近期的Genie模型在多个环境中表现出共享行为能力,但其训练仍依赖昂贵的示范。为此,我们提出仅使用随机代理的训练框架。该方法虽具备良好控制性,但受限于随机探索范围。为此,我们设计了完全依赖世界模型不确定性的AutoExplore代理,可自主获取多样化数据以持续学习。该代理不依赖环境特定奖励,能轻松适配新环境。结合此方法,预训练的多环境模型可快速适应新环境,显著提升视频保真度和可控性。为构建大规模自动交互数据集,我们对974个虚拟环境按行为与控制相似性进行分组,并标注其行为与控制——形成名为RetroAct的数据集。我们还开源了Genie的实现版本GenieRedux,并进一步改进为GenieRedux-G。代码与数据已公开于https://github.com/insait-institute/GenieRedux。
原文摘要 · Abstract (English)
Modern world models require costly and time-consuming collection of large video datasets with action demonstrations by people or by environment-specific agents. To simplify training, we focus on using many virtual environments for inexpensive, automatically collected interaction data. Genie, a recent multi-environment world model, demonstrates simulation abilities of many environments with shared behavior. Unfortunately, training their model requires expensive demonstrations. Therefore, we propose a training framework merely using a random agent in virtual environments. While the model trained in this manner exhibits good controls, it is limited by the random exploration possibilities. To address this limitation, we propose AutoExplore Agent - an exploration agent that entirely relies on the uncertainty of the world model, delivering diverse data from which it can learn the best. Our agent is fully independent of environment-specific rewards and thus adapts easily to new environments. With this approach, the pretrained multi-environment model can quickly adapt to new environments achieving video fidelity and controllability improvement. In order to obtain automatically large-scale interaction datasets for pretraining, we group environments with similar behavior and controls. To this end, we annotate the behavior and controls of 974 virtual environments - a dataset that we name RetroAct. For building our model, we first create an open implementation of Genie - GenieRedux and apply enhancements and adaptations in our version GenieRedux-G. Our code and data are available at https://github.com/insait-institute/GenieRedux.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。