用代码生成环境,让智能体逐步学会复杂技能。
Dreaming in Code for Curriculum Learning in Open-Ended Worlds
- 用基础模型生成可执行代码来构建渐进式学习环境
- 在复杂世界中提升平均回报16%,实现之前失败的高阶任务
- 适合研究持续学习与自动课程设计的学者
开放世界学习将智能视为与不断扩展的环境持续互动中涌现的产物。尽管近期工作利用基础模型自动生成多样化环境,但多聚焦于孤立行为发现,而非持续进展。在复杂的开放世界中,可能挑战的组合空间巨大,导致智能体难以找到始终可学的经验序列。为此,我们提出“代码梦境”(DiCode)框架,通过基础模型合成可执行环境代码,引导学习向更高能力演进。在DiCode中,“梦境”表现为生成世界级别的代码变体。我们在具有丰富机制和长程目标的Craftax基准上实现该框架。实验表明,DiCode使智能体掌握长程技能,在平均回报上比最强基线提升16%,并在先前方法失败的后期战斗任务中获得非零成功率。结果表明,代码级环境设计为课程控制提供了实用机制,可构造中间环境以弥合开放世界中的能力差距。
原文摘要 · Abstract (English)
Open-ended learning frames intelligence as emerging from continual interaction with an ever-expanding space of environments. While recent advances have utilized foundation models to programmatically generate diverse environments, these approaches often focus on discovering isolated behaviors rather than orchestrating sustained progression. In complex open-ended worlds, the large combinatorial space of possible challenges makes it difficult for agents to discover sequences of experiences that remain consistently learnable. To address this, we propose Dreaming in Code (DiCode), a framework in which foundation models synthesize executable environment code to scaffold learning toward increasing competence. In DiCode, "dreaming" takes the form of materializing code-level variations of the world. We instantiate DiCode in Craftax, a challenging open-ended benchmark characterized by rich mechanics and long-horizon progression. Empirically, DiCode enables agents to acquire long-horizon skills, achieving a $16\%$ improvement in mean return over the strongest baseline and non-zero success on late-game combat tasks where prior methods fail. Our results suggest that code-level environment design provides a practical mechanism for curriculum control, enabling the construction of intermediate environments that bridge competence gaps in open-ended worlds. Project page and source code are available at https://konstantinosmitsides.github.io/dreaming-in-code and https://github.com/konstantinosmitsides/dreaming-in-code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。