用代数方法逼近神经网络的隐空间流形,提升效率与泛化能力。
Approximating Latent Manifolds in Neural Networks via Vanishing Ideals
- 通过零理想多项式生成器刻画类别流形结构
- 模型层数减少超半,准确率相当,参数更少,吞吐更高
- 适合追求高效高泛化性能的模型部署场景
深度神经网络通过学习强大的隐表示重塑了现代机器学习,这些表示常符合流形假设:高维数据位于低维流形上。本文首次建立流形学习与计算代数的联系,证明零理想可表征深度网络的隐流形。为此,提出新架构:(i) 在预训练网络中间层截断;(ii) 用零理想的多项式生成器逼近各类别流形;(iii) 通过单个多项式层将隐空间转换为线性可分特征。所提模型层数显著少于基线,准确率相当,参数更少,吞吐更高。结合谱复杂度分析,推导出更优的泛化理论保证,表明本方法原则上优于标准深度网络。数值实验验证其有效性和高效性。
原文摘要 · Abstract (English)
Deep neural networks have reshaped modern machine learning by learning powerful latent representations that often align with the manifold hypothesis: high-dimensional data lie on lower-dimensional manifolds. In this paper, we establish a connection between manifold learning and computational algebra by demonstrating how vanishing ideals can characterize the latent manifolds of deep networks. To that end, we propose a new neural architecture that (i) truncates a pretrained network at an intermediate layer, (ii) approximates each class manifold via polynomial generators of the vanishing ideal, and (iii) transforms the resulting latent space into linearly separable features through a single polynomial layer. The resulting models have significantly fewer layers than their pretrained baselines, while maintaining comparable accuracy, achieving higher throughput, and utilizing fewer parameters. Furthermore, drawing on spectral complexity analysis, we derive sharper theoretical guarantees for generalization, showing that our approach can in principle offer tighter bounds than standard deep networks. Numerical experiments confirm the effectiveness and efficiency of the proposed approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。