通过自抽象分层训练，提升深度网络的稳定性和泛化能力。
Self-Abstraction Learning for Effective and Stable Training of Deep Neural Networks
- 自顶向下逐层训练，用简单网络引导复杂网络
- 在数据稀缺和深层结构下仍保持稳定训练与良好泛化
- 适用于MLP/CNN/RNN，解决梯度消失与过拟合问题
大规模深度神经网络的有效稳定训练对深度学习应用至关重要。然而，传统方法依赖单一大型网络训练，常面临梯度消失、过拟合和学习不稳等问题。为此，我们提出自抽象学习（Self-Abstraction Learning, SAL），一种分层框架：将网络按结构复杂度排列，先训练最顶层的简单网络，其隐藏层与输出层作为后续更复杂网络的指导信号。这种自顶向下的序列引导有效缓解优化难题，实现深层架构的稳定训练。在MLP、CNN和RNN等多种架构上的实验表明，SAL始终优于传统方法，在数据稀疏和复杂网络场景下仍具强鲁棒性与泛化能力。
原文摘要 · Abstract (English)
Training large-scale deep neural networks effectively and stably is essential for applying deep learning across various fields. However, conventional methods, which rely on training a single large network, often encounter challenges such as gradient vanishing, overfitting and unstable learning. To overcome these limitations, we introduce Self-Abstraction Learning (SAL), a hierarchical framework. In SAL, networks are arranged by structural complexity, where the simplest topmost network is trained first and its hidden and output layers serve as guidance for the successively more complex networks below. This top-down sequential guidance effectively mitigates optimization issues, enabling stable training of deep architectures. Various experiments across MLP, CNN, and RNN architectures demonstrate that SAL consistently outperforms conventional methods, ensuring robust generalization even in data-scarce and complex network regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。