用低秩张量分解理论解释深度神经网络的性能机制
Low-Rank Tensor Decompositions for the Theory of Neural Networks
- 通过张量分解与神经网络的数学关联，揭示其表达能力
- 提供多项式时间算法，支持模型可学习性与唯一性分析
- 适合研究深度学习理论的学者与交叉领域研究人员
深度神经网络（NNs）的卓越表现推动了对其理论基础的广泛研究。低秩张量分解因其与神经网络的紧密联系及丰富的理论成果，成为该领域的理想工具。不同张量分解方法具有强唯一性保证，可直接解释其因子，且已有多项式时间算法实现计算。通过张量与神经网络的映射关系，这些结果支撑了深度神经网络理论中的诸多重要进展，涵盖表达能力、算法可学习性与计算难度、泛化性能以及可识别性等方面。本文综述了低秩张量方法在解释深度神经网络性能中的核心作用，系统梳理了来自计算机科学、数学等领域的现有研究，以统一视角呈现其理论价值，并展望其在深度学习理论中的更广泛应用。
原文摘要 · Abstract (English)
The groundbreaking performance of deep neural networks (NNs) promoted a surge of interest in providing a mathematical basis to deep learning theory. Low-rank tensor decompositions are specially befitting for this task due to their close connection to NNs and their rich theoretical results. Different tensor decompositions have strong uniqueness guarantees, which allow for a direct interpretation of their factors, and polynomial time algorithms have been proposed to compute them. Through the connections between tensors and NNs, such results supported many important advances in the theory of NNs. In this review, we show how low-rank tensor methods--which have been a core tool in the signal processing and machine learning communities--play a fundamental role in theoretically explaining different aspects of the performance of deep NNs, including their expressivity, algorithmic learnability and computational hardness, generalization, and identifiability. Our goal is to give an accessible overview of existing approaches (developed by different communities, ranging from computer science to mathematics) in a coherent and unified way, and to open a broader perspective on the use of low-rank tensor decompositions for the theory of deep NNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。