用自监督内在目标模拟斑马鱼自主探索,首次实现无外部数据的脑活动预测。
Intrinsic Goals for Autonomous Agents: Model-Based Exploration in Virtual Zebrafish Predicts Ethological Behavior and Whole-Brain Dynamics
- 基于世界模型与先验的差异构建内在驱动力,实现动物级探索行为
- 在无任务、无奖励条件下生成与真实斑马鱼一致的行为模式和全脑神经胶质动态
- 适用于研究动物自主性、神经计算建模及具身智能系统设计
自主性是动物智能的标志,使其能在复杂环境中无需外部奖励或任务结构即表现出适应性行为。现有强化学习中的无奖赏探索方法,包括一类基于模型的内在动机方法,表现出不一致的探索模式且无法收敛至探索策略,因而无法捕捉动物中观察到的稳健自主行为。此外,系统神经科学大多忽视了自主性的神经基础,更关注由外部奖励驱动的实验范式而非自然状态下的任务无关行为。为弥合这一差距,我们提出一种新型基于模型的内在驱动力(3M-Progress),其设计原则直接源自动物自主探索机制。该方法通过追踪在线世界模型与从生态位中学习的固定先验之间的差异来实现类动物探索。据我们所知,这是首个完全通过自监督优化内在目标而预测脑数据的自主具身智能体,未使用任何行为或神经训练数据。3M-Progress 能够解释斑马鱼幼鱼自主行为模式和全脑神经-胶质动态中的可解释方差,从而提供首个目标驱动、群体层面的神经-胶质计算模型。研究结果建立了一个将基于模型的内在动机与自然行为相联系的计算框架,为构建具有动物级自主性的智能体奠定基础。
原文摘要 · Abstract (English)
Autonomy is a hallmark of animal intelligence, enabling adaptive and intelligent behavior in complex environments without relying on external reward or task structure. Existing reinforcement learning approaches to exploration in reward-free environments, including a class of methods known as model-based intrinsic motivation, exhibit inconsistent exploration patterns and do not converge to an exploratory policy, thus failing to capture robust autonomous behaviors observed in animals. Moreover, systems neuroscience has largely overlooked the neural basis of autonomy, focusing instead on experimental paradigms where animals are motivated by external reward rather than engaging in ethological, naturalistic and task-independent behavior. To bridge these gaps, we introduce a novel model-based intrinsic drive explicitly designed after the principles of autonomous exploration in animals. Our method (3M-Progress) achieves animal-like exploration by tracking divergence between an online world model and a fixed prior learned from an ecological niche. To the best of our knowledge, we introduce the first autonomous embodied agent that predicts brain data entirely from self-supervised optimization of an intrinsic goal -- without any behavioral or neural training data -- demonstrating that 3M-Progress agents capture the explainable variance in behavioral patterns and whole-brain neural-glial dynamics recorded from autonomously behaving larval zebrafish, thereby providing the first goal-driven, population-level model of neural-glial computation. Our findings establish a computational framework connecting model-based intrinsic motivation to naturalistic behavior, providing a foundation for building artificial agents with animal-like autonomy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。