揭示复数空间中实值激活网络的存储容量短板与优势
Shortcomings and capacities of real-constrained neural networks in complex spaces
- 用哈里斯-昌德拉-伊茨克森-祖伯公式分析实激活约束下的存储能力
- 发现实值预激活在临界容量时的存储容量比复值低约30%
- 方法适用于单位阵和正交流形积分，适合研究神经网络几何性质
我们研究了在复数假设类中强制使用实预激活与使用复激活时的存储容量渐近比。方法基于临界容量下的加德纳体积比较，证明依赖于非标准应用的哈里斯-昌德拉-伊茨克森-祖伯（HCIZ）公式。借助该公式，可获得更稳健的最终渐近比估计。这一策略因需在酉群和正交群紧流形上积分而适用，通过韦尔积分公式和哈尔测度实现。
原文摘要 · Abstract (English)
We find the asymptotic ratio between the storage capacities when enforcing real pre-activations in a complex hypothesis class as opposed to complex ones in the same class. Our methods depend on Gardner volume comparisons at critical capacity. Our proof relies on an application of the Harish-Chandra-Itzykson-Zuber (HCIZ) formula, nonstandard in literature. With the HCIZ formula, we may obtain a more robust approximation for the final asymptotic ratio. This strategy is applicable to our work specifically since we integrate over the unitary and orthogonal compact manifolds, facilitated via the Weyl integration formula and the Haar measure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。