用神经滤波器防止生成模型训练时的模型坍缩，无需增加样本量。
Preventing Model Collapse via Contraction-Conditioned Neural Filters
- 设计基于压缩算子的神经滤波器，主动学习满足收缩条件。
- 理论证明：固定样本量下误差概率收敛至零。
- 实验验证：在固定数据量下有效阻止模型坍缩，适合实际部署。
本文提出一种基于压缩算子的神经网络滤波方法，用于解决生成模型递归训练中的模型坍缩问题。与需超线性样本增长（$O(t^{1+s})$）的方法不同，本方法在无偏估计框架下完全消除对样本量增加的依赖，通过设计可学习收缩条件的神经滤波器实现。我们构建了专用神经架构与损失函数，使滤波器能主动满足指数族分布下的假设2.3的收缩条件，从而保障理论结果的实际应用。理论分析表明，当学习到的收缩条件满足时，即使样本量恒定，估计误差仍可概率收敛，即对任意 $δ>0$，有 $\ ext{limsup}_{t\to\infty}\mathbb{P}(\|\mathbf{e}_t\|>δ)=0$。实验结果表明，该神经滤波器能有效学习收缩条件，在固定样本量设置下成功防止模型坍缩，为实际应用提供端到端解决方案。
原文摘要 · Abstract (English)
This paper presents a neural network filter method based on contraction operators to address model collapse in recursive training of generative models. Unlike \cite{xu2024probabilistic}, which requires superlinear sample growth ($O(t^{1+s})$), our approach completely eliminates the dependence on increasing sample sizes within an unbiased estimation framework by designing a neural filter that learns to satisfy contraction conditions. We develop specialized neural network architectures and loss functions that enable the filter to actively learn contraction conditions satisfying Assumption 2.3 in exponential family distributions, thereby ensuring practical application of our theoretical results. Theoretical analysis demonstrates that when the learned contraction conditions are satisfied, estimation errors converge probabilistically even with constant sample sizes, i.e., $\limsup_{t\to\infty}\mathbb{P}(\|\mathbf{e}_t\|>δ)=0$ for any $δ>0$. Experimental results show that our neural network filter effectively learns contraction conditions and prevents model collapse under fixed sample size settings, providing an end-to-end solution for practical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。