揭示生成模型反馈循环中的神经共振现象，解释模型崩溃机制。
A Markovian View of Iterative-Feedback Loops in Image Generative Models: Neural Resonance and Model Collapse
- 将反馈过程建模为马尔可夫链，发现收敛到低维不变结构。
- 在MNIST、ImageNet等数据集上验证了八类崩溃模式，发现方向性收缩是关键条件。
- 提出诊断工具，适用于扩散模型、CycleGAN等生成系统，指导防崩溃设计。
AI训练数据不可避免包含由AI生成的内容，导致模型输出影响后续训练的反馈循环。此类迭代反馈可能引发模型崩溃，但其内在机制尚不清晰。本文表明，广泛存在的反馈过程会收敛至潜在空间中的低维不变结构，称为神经共振。通过将迭代反馈建模为马尔可夫链，我们证明产生共振需满足两个条件：反馈过程的遍历性与潜在表示的方向性收缩。通过对扩散模型在MNIST和ImageNet上的实验，以及CycleGAN和音频反馈实验，我们追踪了局部与全局流形几何的变化，并提出八类崩溃行为的分类体系。神经共振为生成模型长期退化行为提供了统一解释，并可作为实际诊断工具，用于识别、刻画并最终缓解模型崩溃。
原文摘要 · Abstract (English)
AI training datasets will inevitably contain AI-generated examples, leading to ``feedback'' in which the output of one model impacts the training of another. It is known that such iterative feedback can lead to model collapse, yet the mechanisms underlying this degeneration remain poorly understood. Here we show that a broad class of feedback processes converges to a low-dimensional invariant structure in latent space, a phenomenon we call neural resonance. By modeling iterative feedback as a Markov Chain, we show that two conditions are needed for this resonance to occur: ergodicity of the feedback process and directional contraction of the latent representation. By studying diffusion models on MNIST and ImageNet, as well as CycleGAN and an audio feedback experiment, we map how local and global manifold geometry evolve, and we introduce an eight-pattern taxonomy of collapse behaviors. Neural resonance provides a unified explanation for long-term degenerate behavior in generative models and provides practical diagnostics for identifying, characterizing, and eventually mitigating collapse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。