arXiv:2503.21794cs.NEcs.AI2025-03

用能量视角重新理解神经网络,揭示信息本质与自组织机制

Architecture of Information

  • 将神经网络建模为能量系统,通过熵与焓分析其内部结构
  • 发现信息熵具有能量属性,输出信息对应焓的有序部分
  • 提出无需外部算法的直接学习新框架,适合理论研究者

本文探讨了构建形式化神经元及多层人工神经网络(ANN)能量景观的方法。分析揭示了分类型ANN(如MLP或CNN)和生成型ANN模型的概念局限性。通过对形式化神经元和ANN模型中信息熵与热力学熵的研究,得出信息熵具有能量本质的结论。应用吉布斯自由能概念,可将ANN的输出信息视为焓的有序部分。将ANN建模为能量系统,使其内能结构可解释为对外部世界的内部模型,该模型基于内部能量组分的相互作用实现自组织。通过基于约简算子的能量函数(类同于李雅普诺夫函数)控制这一自组织与演化过程,从而提出一种无需额外外部算法的直接学习新方法,适用于自组织与进化型ANN的构建。本研究还首次从系统内外能量相互作用的角度,给出了信息的正式定义。

原文摘要 · Abstract (English)

The paper explores an approach to constructing energy landscapes of a formal neuron and multilayer artificial neural networks (ANNs). Their analysis makes it possible to determine the conceptual limitations of both classification ANNs (e.g., MLP or CNN) and generative ANN models. The study of informational and thermodynamic entropy in formal neuron and ANN models leads to the conclusion about the energetic nature of informational entropy. The application of the Gibbs free energy concept allows representing the output information of ANNs as the structured part of enthalpy. Modeling ANNs as energy systems makes it possible to interpret the structure of their internal energy as an internal model of the external world, which self-organizes based on the interaction of the system's internal energy components. The control of the self-organization and evolution process of this model is carried out through an energy function (analogous to the Lyapunov function) based on reduction operators. This makes it possible to introduce a new approach to constructing self-organizing and evolutionary ANNs with direct learning, which does not require additional external algorithms. The presented research makes it possible to formulate a formal definition of information in terms of the interaction processes between the internal and external energy of the system.

神经网络能量模型信息熵自组织

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。