从自由能原理推导出自组织吸引子网络,无需预设规则即可实现高效学习与推理。
Self-orthogonalizing attractor neural networks emerging from the free energy principle
- 基于自由能原理,无需人工设定规则,自动形成吸引子动力学。
- 吸引子呈近正交分布,提升对输入空间的覆盖和泛化能力。
- 适用于神经科学与人工智能中的自组织系统建模,具生物合理性。
吸引子动力学是复杂系统(如大脑)的核心特征。本文通过将自由能原理应用于随机动力系统的通用划分,形式化推导出吸引子网络的涌现机制。该方法无需显式设定学习与推理规则,自然生成高效且符合生物学的推断与学习动态,形成多层级贝叶斯主动推断过程:自由能景观上的吸引子编码先验信念,推断将感官数据融合为后验信念,学习则优化连接权重以最小化长期意外。理论分析与仿真表明,所提网络倾向于产生近正交化的吸引子表示,这是同时优化预测精度与模型复杂度的结果。这些吸引子高效覆盖输入子空间,增强泛化能力及隐变量与可观测效应间的互信息。随机数据下生成对称稀疏连接,而序列数据则催生非对称连接与非平衡稳态动力学,自然推广了传统玻尔兹曼机。本研究为自组织吸引子网络提供统一理论框架,为人工智能与神经科学带来新洞见。
原文摘要 · Abstract (English)
Attractor dynamics are a hallmark of many complex systems, including the brain. Understanding how such self-organizing dynamics emerge from first principles is crucial for advancing our understanding of neuronal computations and the design of artificial intelligence systems. Here we formalize how attractor networks emerge from the free energy principle applied to a universal partitioning of random dynamical systems. Our approach obviates the need for explicitly imposed learning and inference rules and identifies emergent, but efficient and biologically plausible inference and learning dynamics for such self-organizing systems. These result in a collective, multi-level Bayesian active inference process. Attractors on the free energy landscape encode prior beliefs; inference integrates sensory data into posterior beliefs; and learning fine-tunes couplings to minimize long-term surprise. Analytically and via simulations, we establish that the proposed networks favor approximately orthogonalized attractor representations, a consequence of simultaneously optimizing predictive accuracy and model complexity. These attractors efficiently span the input subspace, enhancing generalization and the mutual information between hidden causes and observable effects. Furthermore, while random data presentation leads to symmetric and sparse couplings, sequential data fosters asymmetric couplings and non-equilibrium steady-state dynamics, offering a natural generalization of conventional Boltzmann Machines. Our findings offer a unifying theory of self-organizing attractor networks, providing novel insights for AI and neuroscience.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。