证明可逆残差网络仍受维度诅咒，采样复杂度难降。
The sampling complexity of learning invertible residual neural networks
- 研究可逆残差网络的点采样学习复杂度
- 发现其统一范数逼近仍受维度诅咒影响
- 适合关注深度生成模型理论边界的研究者
近期研究表明，从点采样中以高一致精度确定前馈ReLU神经网络会遭遇维度诅咒，导致样本需求量随维度指数增长，限制了其在需高一致精度场景的应用。本文探讨通过限制网络架构是否能改善采样复杂度。研究对象为深度学习基础架构——可逆残差神经网络，其广泛应用于现代生成模型。核心结论表明：残差结构与可逆性并不能克服简单前馈架构所面临的复杂度障碍。具体而言，从点采样中以统一范数逼近可逆残差神经网络的计算复杂度依然遭受维度诅咒。类似结果也适用于可逆卷积残差神经网络。
原文摘要 · Abstract (English)
In recent work it has been shown that determining a feedforward ReLU neural network to within high uniform accuracy from point samples suffers from the curse of dimensionality in terms of the number of samples needed. As a consequence, feedforward ReLU neural networks are of limited use for applications where guaranteed high uniform accuracy is required. We consider the question of whether the sampling complexity can be improved by restricting the specific neural network architecture. To this end, we investigate invertible residual neural networks which are foundational architectures in deep learning and are widely employed in models that power modern generative methods. Our main result shows that the residual neural network architecture and invertibility do not help overcome the complexity barriers encountered with simpler feedforward architectures. Specifically, we demonstrate that the computational complexity of approximating invertible residual neural networks from point samples in the uniform norm suffers from the curse of dimensionality. Similar results are established for invertible convolutional Residual neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。