对比人与智能体在开放世界中的探索行为，发现熵和自主性更贴近人类探索机制。
Intrinsically-Motivated Humans and Agents in Open-World Exploration
- 通过比较人类与智能体在Crafter环境中的探索，分析熵、信息增益、自主性三种内在目标
- 熵和自主性与人类探索进展正相关，且熵先快速上升后平稳，自主性持续增长
- 儿童探索中自言自语尤其是目标表述可能促进探索，适合认知科学与强化学习交叉研究
探索的驱动力是什么？理解内在动机是认知科学与人工智能领域长期挑战；尽管已有多种目标被用于训练智能体，但人类与智能体的探索行为仍存在差距。我们直接在复杂开放式环境Crafter中比较成年人、儿童与AI智能体的探索行为，研究熵、信息增益、自主性等常见内在目标与其行为的关系。结果表明，仅有熵和自主性与人类探索进展呈一致正相关，暗示它们更适合作为智能体内在奖励设计的依据。此外，在人类与智能体中均观察到：熵初始快速上升后趋于平稳，而自主性持续增长，提示状态多样性在早期探索中提供更强信号，而高级探索应更注重控制能力。最后，初步发现儿童的私密言语，特别是目标陈述，可能有助于探索。数据可在https://github.com/alyd/humans_in_crafter_data获取。
原文摘要 · Abstract (English)
What drives exploration? Understanding intrinsic motivation is a long-standing challenge in both cognitive science and artificial intelligence; numerous objectives have been proposed and used to train agents, yet there remains a gap between human and agent exploration. We directly compare adults, children, and AI agents in a complex open-ended environment, Crafter, and study how common intrinsic objectives: Entropy, Information Gain, and Empowerment, relate to their behavior. We find that only Entropy and Empowerment are consistently positively correlated with human exploration progress, indicating that these objectives may better inform intrinsic reward design for agents. Furthermore, across agents and humans we observe that Entropy initially increases rapidly, then plateaus, while Empowerment increases continuously, suggesting that state diversity may provide more signal in early exploration, while advanced exploration should prioritize control. Finally, we find preliminary evidence that private speech utterances, and particularly goal verbalizations, may aid exploration in children. Our data is available at https://github.com/alyd/humans_in_crafter_data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。