提出Pool Skip结构,解决深层网络训练中的节点失效问题。
Beyond Skip Connection: Pooling and Unpooling Design for Elimination Singularities
- 融合池化、反池化与残差连接,稳定特征传播
- 在2D和3D图像任务中提升分类与分割性能
- 理论支持:权重惯性假说解释结构有效性
深度卷积神经网络(CNN)训练面临消除奇点的挑战,即节点持续失活导致损失曲面退化。为此,我们提出Pool Skip架构,通过结合最大池化、最大反池化、3×3卷积和残差连接,增强训练稳定性并保持特征完整性。同时提出权重惯性假说,从维度与仿射补偿角度提供理论支撑。在多种基准测试中,包括2D自然图像与3D医学影像的分类与分割任务,验证了该方法在提升模型鲁棒性与性能方面的有效性。
原文摘要 · Abstract (English)
Training deep Convolutional Neural Networks (CNNs) presents unique challenges, including the pervasive issue of elimination singularities, consistent deactivation of nodes leading to degenerate manifolds within the loss landscape. These singularities impede efficient learning by disrupting feature propagation. To mitigate this, we introduce Pool Skip, an architectural enhancement that strategically combines a Max Pooling, a Max Unpooling, a 3 times 3 convolution, and a skip connection. This configuration helps stabilize the training process and maintain feature integrity across layers. We also propose the Weight Inertia hypothesis, which underpins the development of Pool Skip, providing theoretical insights into mitigating degradation caused by elimination singularities through dimensional and affine compensation. We evaluate our method on a variety of benchmarks, focusing on both 2D natural and 3D medical imaging applications, including tasks such as classification and segmentation. Our findings highlight Pool Skip's effectiveness in facilitating more robust CNN training and improving model performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。