证明多项式复合的线性无关性，揭示深度网络参数可识别性机制
Linear Independence of Polynomial Compositions and Identifiability of Deep Neural Networks
- 通过多项式复合构造线性无关函数，推广经典多项式幂定理
- 在双多项式及低度多多项式情形下证明猜想成立
- 为具有通用多项式激活函数的深层网络提供参数对称性完整刻画
针对深度学习中的理论问题，我们猜想：将固定数量互异的非平凡多项式与一个足够高次的通用多项式进行后复合，所得多项式集合是线性无关的。该猜想推广了 Newman--Slater 关于多项式幂的定理。本文证明了该猜想的若干情形及其传递性变体：对两个多项式的情形给出证明；对任意数量多项式但其次数有界的情形也成立。此外，我们证明该猜想可推出具有通用多项式激活函数的深层全连接神经网络架构的完全可识别性（即参数对称性）。特别地，对于各层激活函数次数递增的网络结构，本文已建立的猜想版本能完整刻画所有产生相同端到端函数的参数集合。作为特例，我们彻底解决了浅层多项式网络的可识别性问题。
原文摘要 · Abstract (English)
Motivated by theoretical problems in deep learning, we conjecture that post-composing a fixed number of pairwise distinct nonconstant polynomials with a generic polynomial of sufficiently large degree yields linearly independent polynomials. This generalizes Newman--Slater's theorem on powers of polynomials. We establish several cases of this conjecture and its origin-passing variant: We prove the result for two polynomials, and for an arbitrary number of polynomials when their degrees are bounded. Furthermore, we show how the conjecture implies a complete understanding of the identifiability (i.e., parameter symmetries) of deep fully connected neural network architectures with generic polynomial activation functions. In particular, for network architectures with layer-specific activations of increasing degree, our established versions of the conjecture fully characterize the set of parameters yielding the same end-to-end network function. As a special case, we fully resolve the identifiability of shallow polynomial networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。