arXiv:2511.14329cs.CV2025-11被引 3

提出StepsNet架构,解决深层网络训练难题,提升模型性能。

Step by Step Network

  • 将特征按通道分层,逐块递增宽度,逐步学习
  • 在图像分类等任务中显著优于传统残差网络
  • 适合需要深度扩展的各类视觉与语言模型

提升网络深度是神经架构设计的核心目标,理论表明更深的模型具备指数级增强的能力。得益于残差连接,现代神经网络已可达到百层以上并取得广泛应用。然而,随着网络持续加深,现有架构难以充分发挥其理论潜力,亟需更先进的设计以释放深层网络的性能。本文识别出两个阻碍残差模型进一步深化的关键因素:捷径退化和宽度受限。捷径退化抑制深层学习,而固有的深度-宽度权衡限制了模型宽度。为此,我们提出一种广义残差架构——步进网络(Step by Step Network, StepsNet),通过沿通道维度分离特征,并堆叠宽度递增的模块实现渐进式学习。该方法有效缓解上述问题,可作为通用宏观设计适配多种模型。大量实验表明,该方法在图像分类、目标检测、语义分割及语言建模等任务中均一致超越传统残差网络,验证了StepsNet作为广泛采用的残差架构的优越泛化能力。

原文摘要 · Abstract (English)

Scaling up network depth is a fundamental pursuit in neural architecture design, as theory suggests that deeper models offer exponentially greater capability. Benefiting from the residual connections, modern neural networks can scale up to more than one hundred layers and enjoy wide success. However, as networks continue to deepen, current architectures often struggle to realize their theoretical capacity improvements, calling for more advanced designs to further unleash the potential of deeper networks. In this paper, we identify two key barriers that obstruct residual models from scaling deeper: shortcut degradation and limited width. Shortcut degradation hinders deep-layer learning, while the inherent depth-width trade-off imposes limited width. To mitigate these issues, we propose a generalized residual architecture dubbed Step by Step Network (StepsNet) to bridge the gap between theoretical potential and practical performance of deep models. Specifically, we separate features along the channel dimension and let the model learn progressively via stacking blocks with increasing width. The resulting method mitigates the two identified problems and serves as a versatile macro design applicable to various models. Extensive experiments show that our method consistently outperforms residual models across diverse tasks, including image classification, object detection, semantic segmentation, and language modeling. These results position StepsNet as a superior generalization of the widely adopted residual architecture.

深度网络残差学习架构设计性能提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。