用熵的分层最小化实现神经网络自主学习，无需反向传播。
Structured Knowledge Accumulation: An Autonomous Framework for Layer-Wise Entropy Reduction in Neural Learning
- 将熵视为各层知识对决策概率的影响，定义为可优化的动态量
- 各层独立优化使整体熵逐层下降，知识结构渐进演化
- 适合资源受限或并行计算场景，具生物合理性
我们提出结构化知识累积（SKA）框架，将熵重新诠释为神经网络中分层的知识对齐动态度量。不同于传统基于梯度的优化，SKA以知识向量及其对多层决策概率的影响来定义熵。该形式自然引出sigmoid等激活函数作为熵最小化的结果。与常规反向传播不同，SKA允许每层独立优化，通过与决策概率变化对齐来调整自身知识表示。结果是网络总熵以层级方式递减，知识结构得以逐步演化。该方法提供了一种可扩展、具生物合理性的梯度学习替代方案，连接信息论与人工智能，在资源受限和并行计算环境中具有广阔应用前景。
原文摘要 · Abstract (English)
We introduce the Structured Knowledge Accumulation (SKA) framework, which reinterprets entropy as a dynamic, layer-wise measure of knowledge alignment in neural networks. Instead of relying on traditional gradient-based optimization, SKA defines entropy in terms of knowledge vectors and their influence on decision probabilities across multiple layers. This formulation naturally leads to the emergence of activation functions such as the sigmoid as a consequence of entropy minimization. Unlike conventional backpropagation, SKA allows each layer to optimize independently by aligning its knowledge representation with changes in decision probabilities. As a result, total network entropy decreases in a hierarchical manner, allowing knowledge structures to evolve progressively. This approach provides a scalable, biologically plausible alternative to gradient-based learning, bridging information theory and artificial intelligence while offering promising applications in resource-constrained and parallel computing environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。