用熵空间理论为深度学习提供数学基础框架
Understanding Deep Learning via Entropy Space Theory
- 构建独立于参数的熵空间，通过拓扑结构覆盖模型所有可能状态
- 证明熵空间是满足公理化体系的赋范空间，具备数学严谨性
- 提出统一坐标系，按信息熵最大值压缩程度排序模型状态
深度学习常因理论研究滞后于实践而受到质疑。本文首次引入熵空间理论，该空间通过拓扑结构可覆盖任意深度学习模型的所有可能状态，且独立于网络参数。通过设计基本运算与范数，证明熵空间在形式公理框架下是一个赋范空间。基于此理论，提出统一坐标系，可对模型的每一状态进行坐标表示，并按信息熵最大值的压缩程度进行排序。该理论为深度学习的数学基础提供了新颖的先验框架。
原文摘要 · Abstract (English)
Deep learning is often criticized for its theoretical research lagging behind practice. To make deep learning easier to understand, the entropy space theory is first introduced here. The entropy space can cover all the possibilities of any deep learning model by topological structure. It is independent of network parameters. Through the designed fundamental operations and norm, entropy space is proven to be a normed space within the formal axiomatic framework. Based on the theory, a unified coordinate system is proposed. It can coordinatize every state of a model and rank them by compression of the maximal value of information entropy. The theory offers a novel priori framework for mathematical fundamentals of deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。