揭示深度神经网络中'欺骗样本'的动态机制，提出新型解法突破模型退化瓶颈。
Wormhole Dynamics in Deep Neural Networks
- 基于最大似然框架分析模型输出空间坍缩现象
- 发现层数过多导致模型退化至映射所有输入为相同输出
- 提出'虫洞'解法可重构随机输入与标签的合理关联，适用于无监督学习研究
本文研究深度神经网络（DNN）的泛化行为，聚焦于'欺骗样本'现象——即模型对人类看来随机或无结构的输入仍能自信分类。为此，我们提出一种基于最大似然估计的解析框架，不依赖传统的梯度优化与显式标签。分析发现，过度参数化的DNN在输出特征空间出现坍缩，虽提升泛化能力，但增加层数后会进入退化状态：模型通过将不同输入映射到相同输出实现零损失。进一步研究发现，采用新提出的'虫洞'解法可避免该退化。该解法在任意欺骗样本上可调和随机输入与有意义标签间的矛盾，为捷径学习提供新视角。研究深化了对DNN泛化机制的理解，并指明未来在无监督学习中探索学习动力学的方向。
原文摘要 · Abstract (English)
This work investigates the generalization behavior of deep neural networks (DNNs), focusing on the phenomenon of "fooling examples," where DNNs confidently classify inputs that appear random or unstructured to humans. To explore this phenomenon, we introduce an analytical framework based on maximum likelihood estimation, without adhering to conventional numerical approaches that rely on gradient-based optimization and explicit labels. Our analysis reveals that DNNs operating in an overparameterized regime exhibit a collapse in the output feature space. While this collapse improves network generalization, adding more layers eventually leads to a state of degeneracy, where the model learns trivial solutions by mapping distinct inputs to the same output, resulting in zero loss. Further investigation demonstrates that this degeneracy can be bypassed using our newly derived "wormhole" solution. The wormhole solution, when applied to arbitrary fooling examples, reconciles meaningful labels with random ones and provides a novel perspective on shortcut learning. These findings offer deeper insights into DNN generalization and highlight directions for future research on learning dynamics in unsupervised settings to bridge the gap between theory and practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。