arXiv:2504.03214cs.LG2025-04

将神经网络学习视为连续时间演化过程,用熵最小作用原理实现自组织训练。

Structured Knowledge Accumulation: The Principle of Entropic Least Action in Forward-Only Neural Learning

  • 把学习率看作连续系统的时步,实现离散优化向连续演化的转变。
  • 发现学习速率与迭代步数乘积恒定时,学习动态保持不变,存在内在时间尺度。
  • 提出熵与知识流收敛作为自然停止条件,适合追求生物启发的高效模型设计。

本文旨在扩展近期提出的结构化知识累积(SKA)框架。引入两个核心概念:张量网络函数和神经学习的特征时间性质。首先,将学习率重新解释为连续系统中的时间步长,使神经学习从离散优化转变为连续时间演化。我们证明当学习率与迭代步数乘积保持恒定时,学习动态保持一致,揭示了时间不变性并识别出网络的内在时间尺度。其次,定义张量网络函数作为衡量决策概率、熵梯度与知识变化关系的指标,并将其零交叉点定义为决策概率与熵梯度之间的平衡状态。我们表明熵与知识流的收敛提供了自然的停止条件,替代了任意阈值,采用信息论准则。此外,我们建立了SKA动力学满足基于欧拉-拉格朗日方程的变分原理。这些发现将SKA拓展为连续且自组织的学习模型,将计算学习与遵循自然定律的物理系统相联系。通过将学习理解为基于时间的过程,为构建高效、鲁棒且生物启发的人工智能系统开辟新方向。

原文摘要 · Abstract (English)

This paper aims to extend the Structured Knowledge Accumulation (SKA) framework recently proposed by \cite{mahi2025ska}. We introduce two core concepts: the Tensor Net function and the characteristic time property of neural learning. First, we reinterpret the learning rate as a time step in a continuous system. This transforms neural learning from discrete optimization into continuous-time evolution. We show that learning dynamics remain consistent when the product of learning rate and iteration steps stays constant. This reveals a time-invariant behavior and identifies an intrinsic timescale of the network. Second, we define the Tensor Net function as a measure that captures the relationship between decision probabilities, entropy gradients, and knowledge change. Additionally, we define its zero-crossing as the equilibrium state between decision probabilities and entropy gradients. We show that the convergence of entropy and knowledge flow provides a natural stopping condition, replacing arbitrary thresholds with an information-theoretic criterion. We also establish that SKA dynamics satisfy a variational principle based on the Euler-Lagrange equation. These findings extend SKA into a continuous and self-organizing learning model. The framework links computational learning with physical systems that evolve by natural laws. By understanding learning as a time-based process, we open new directions for building efficient, robust, and biologically-inspired AI systems.

神经网络学习机制熵原理连续学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。