用图论解释丢弃法：发现能泛化的子网络像连通集群。
A Combinatorial Theory of Dropout: Subnetworks, Graph Geometry, and Generalization
- 将丢弃法建模为在二值子网图上的随机游走
- 泛化能力强的子网形成低电阻连通簇，数量随宽度指数增长
- 适合研究模型正则化与子网优化的研究者
我们提出一种组合与图论视角下的丢弃法理论，将训练过程建模为在二值子网构成的高维图上的随机游走。每个节点代表一个掩码后的网络版本，丢弃法驱动其在图中随机遍历。定义子网贡献度量以量化泛化能力，发现该度量在图上平滑变化。结合谱图论、PAC-Bayes分析与组合数学，证明具有良好泛化性的子网构成大而连通、低电阻的集群，且其数量随网络宽度呈指数级增长。这揭示了丢弃法本质是通过内置冗余从一组鲁棒的、结构化的优良子网中采样。大量实验验证了所有理论假设，涵盖多种架构。结果为理解丢弃法提供了统一框架，并为掩码引导的正则化与子网优化指明新方向。
原文摘要 · Abstract (English)
We propose a combinatorial and graph-theoretic theory of dropout by modeling training as a random walk over a high-dimensional graph of binary subnetworks. Each node represents a masked version of the network, and dropout induces stochastic traversal across this space. We define a subnetwork contribution score that quantifies generalization and show that it varies smoothly over the graph. Using tools from spectral graph theory, PAC-Bayes analysis, and combinatorics, we prove that generalizing subnetworks form large, connected, low-resistance clusters, and that their number grows exponentially with network width. This reveals dropout as a mechanism for sampling from a robust, structured ensemble of well-generalizing subnetworks with built-in redundancy. Extensive experiments validate every theoretical claim across diverse architectures. Together, our results offer a unified foundation for understanding dropout and suggest new directions for mask-guided regularization and subnetwork optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。