用强化学习智能体生成数据，让世界模型更真实地模拟复杂环境。
Learning Generative Interactive Environments By Trained Agent Exploration
- 用训练过的智能体替代随机探索，生成更丰富的环境数据。
- 在Coinrun任务中，新方法显著提升视觉质量和动作可控性。
- 代码开源，适合研究世界模型与强化学习融合的开发者。
世界模型在理解与模拟复杂环境规则和行为方面日益重要。Genie 模型虽能从视觉多样的环境中学习，但依赖昂贵的人工采集数据。我们发现其替代方案——使用随机智能体探索，受限于探索能力。为此，提出采用基于强化学习的智能体生成数据，构建更具多样性数据集，从而提升模型在多种场景及真实动作下的适应性和表现。本文首次发布 GenieRedux——基于 Genie 的实现，并引入变体 GenieRedux-G，利用智能体已有的动作以消除验证阶段的动作预测不确定性。评估包括复制 Coinrun 案例研究，结果表明，GenieRedux-G 在训练智能体探索下实现了更优的视觉保真度与可控性。该方法可复现、可扩展，适用于新型环境。代码库已公开于 https://github.com/insait-institute/GenieRedux。
原文摘要 · Abstract (English)
World models are increasingly pivotal in interpreting and simulating the rules and actions of complex environments. Genie, a recent model, excels at learning from visually diverse environments but relies on costly human-collected data. We observe that their alternative method of using random agents is too limited to explore the environment. We propose to improve the model by employing reinforcement learning based agents for data generation. This approach produces diverse datasets that enhance the model's ability to adapt and perform well across various scenarios and realistic actions within the environment. In this paper, we first release the model GenieRedux - an implementation based on Genie. Additionally, we introduce GenieRedux-G, a variant that uses the agent's readily available actions to factor out action prediction uncertainty during validation. Our evaluation, including a replication of the Coinrun case study, shows that GenieRedux-G achieves superior visual fidelity and controllability using the trained agent exploration. The proposed approach is reproducable, scalable and adaptable to new types of environments. Our codebase is available at https://github.com/insait-institute/GenieRedux .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。