arXiv:2511.12742cs.LG2025-11AAAI被引 4

通过过滤低质量生成数据,防止模型迭代中性能崩溃。

Stabilizing Self-Consuming Diffusion Models with Latent Space Filtering

  • 在潜在空间识别并剔除不真实生成数据,缓解自消耗循环问题。
  • 在多个真实数据集上优于基线方法,有效抑制模型坍塌。
  • 无需额外标注或计算成本,适合大规模生成模型训练场景。

随着互联网上合成数据泛滥,这些数据常被用于训练下一代生成模型,形成‘自消耗循环’,可能导致训练不稳定或模型坍塌。现有解决策略如累积历史数据或注入真实数据,分别增加计算开销或依赖人工标注。本文实证分析了自消耗扩散模型的潜在空间动态,发现合成数据提取的低维潜在表示随代际递减。基于此,提出潜在空间过滤(LSF)方法,通过过滤混合数据集中不真实的合成数据来缓解模型坍塌。理论层面,构建框架将潜在空间退化与实证观察相联系。实验表明,LSF在多个真实数据集上持续优于基线方法,有效缓解模型坍塌,且无需增加训练成本或人工标注。

原文摘要 · Abstract (English)

As synthetic data proliferates across the Internet, it is often reused to train successive generations of generative models. This creates a ``self-consuming loop" that can lead to training instability or \textit{model collapse}. Common strategies to address the issue -- such as accumulating historical training data or injecting fresh real data -- either increase computational cost or require expensive human annotation. In this paper, we empirically analyze the latent space dynamics of self-consuming diffusion models and observe that the low-dimensional structure of latent representations extracted from synthetic data degrade over generations. Based on this insight, we propose \textit{Latent Space Filtering} (LSF), a novel approach that mitigates model collapse by filtering out less realistic synthetic data from mixed datasets. Theoretically, we present a framework that connects latent space degradation to empirical observations. Experimentally, we show that LSF consistently outperforms existing baselines across multiple real-world datasets, effectively mitigating model collapse without increasing training cost or relying on human annotation.

扩散模型生成模型数据过滤

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。