探究智能体如何在好奇与掌控间平衡，实现高效探索。
From Curiosity to Competence: How World Models Interact with the Dynamics of Exploration
- 用世界模型构建内部表征，动态调节好奇心与能力的探索策略。
- 梦境代理（Dreamer）展现探索与表征学习相互促进的双向反馈。
- 适合研究认知机制与强化学习中自适应探索的学者参考。
智能体在探索世界时如何同时保持对环境的控制？从儿童游戏到实验室科学家，智能体必须在好奇心（寻求知识的驱动力）与能力（掌握和控制环境的驱动力）之间取得平衡。本文将内在动机的认知理论与强化学习结合，探讨演化中的内部表征如何调和好奇心（新颖性或信息增益）与能力（自主性）之间的权衡。我们比较了两种基于模型的智能体：使用手工设计状态抽象的表格式（Tabular）代理，以及学习内部世界模型的梦境代理（Dreamer）。表格式代理表现出好奇心与能力引导探索的不同模式，而两者并重则提升了探索效率。梦境代理揭示了探索与表征学习之间的双向互动，反映了好奇心与能力在发展中的协同进化。研究结果将自适应探索形式化为追求未知与可控性的平衡，为认知理论和高效的强化学习提供了新见解。
原文摘要 · Abstract (English)
What drives an agent to explore the world while also maintaining control over the environment? From a child at play to scientists in the lab, intelligent agents must balance curiosity (the drive to seek knowledge) with competence (the drive to master and control the environment). Bridging cognitive theories of intrinsic motivation with reinforcement learning, we ask how evolving internal representations mediate the trade-off between curiosity (novelty or information gain) and competence (empowerment). We compare two model-based agents using handcrafted state abstractions (Tabular) or learning an internal world model (Dreamer). The Tabular agent shows curiosity and competence guide exploration in distinct patterns, while prioritizing both improves exploration. The Dreamer agent reveals a two-way interaction between exploration and representation learning, mirroring the developmental co-evolution of curiosity and competence. Our findings formalize adaptive exploration as a balance between pursuing the unknown and the controllable, offering insights for cognitive theories and efficient reinforcement learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。