arXiv:2601.19958stat.MLcs.LG2026-01

用迭代函数系统理论分析深度网络,给出生成模型泛化界与新训练目标。

Deep Neural Networks as Iterated Function Systems and a Generalization Bound

  • 将深度网络视为随机迭代函数系统,建立数学统一框架。
  • 在合约条件下证明不变测度存在唯一,并导出Wasserstein泛化界。
  • 提出新训练目标,直接优化数据分布与生成分布的逼近误差。

深度神经网络在众多任务中表现卓越,但其数学分析仍碎片化:稳定性与泛化性通常在不同框架下分别研究,且多为个案分析。架构上,深度网络依赖参数化函数的递归应用,该机制易不稳定且难训练,稳定性是核心挑战。即使训练成功,现有对模型在观测数据外泛化能力的严格结果仍很少,尤其在生成建模场景中。本文利用随机迭代函数系统(Stochastic IFS)理论,表明两种重要深度架构可被视作或规范关联于位置依赖的IFS。这一联系使我们能够引入随机动力系统成果:(i) 在合适合约假设下,确立不变测度的存在性与唯一性;(ii) 推导生成建模的Wasserstein泛化界。该界自然引出一种新训练目标,直接控制数据分布与其在学习转移算子下的像之间的类柯莱奇逼近误差。我们在二维可控例子中验证理论,并在标准图像数据集(MNIST、CelebA、CIFAR-10)上实证评估该目标的有效性。

原文摘要 · Abstract (English)

Deep neural networks (DNNs) achieve remarkable performance on a wide range of tasks, yet their mathematical analysis remains fragmented: stability and generalization are typically studied in disparate frameworks and on a case-by-case basis. Architecturally, DNNs rely on the recursive application of parametrized functions, a mechanism that can be unstable and difficult to train, making stability a primary concern. Even when training succeeds, there are few rigorous results on how well such models generalize beyond the observed data, especially in the generative setting. In this work, we leverage the theory of stochastic Iterated Function Systems (IFS) and show that two important deep architectures can be viewed as, or canonically associated with, place-dependent IFS. This connection allows us to import results from random dynamical systems to (i) establish the existence and uniqueness of invariant measures under suitable contractivity assumptions, and (ii) derive a Wasserstein generalization bound for generative modeling. The bound naturally leads to a new training objective that directly controls the collage-type approximation error between the data distribution and its image under the learned transfer operator. We illustrate the theory on a controlled 2D example and empirically evaluate the proposed objective on standard image datasets (MNIST, CelebA, CIFAR-10).

深度网络泛化界生成模型随机动力系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。