受大脑结构启发，用分层残差连接提升深度网络的准确率和训练速度。
Hierarchical Residuals Exploit Brain-Inspired Compositionality
- 在不同层级间建立长程残差连接，模仿哺乳动物大脑的神经连接方式。
- 在多个架构上实现准确率提升与更快收敛，验证了分层组合性机制。
- 适合研究神经网络结构设计与生物启发模型的读者关注。
我们提出分层残差网络（HiResNets），一种具有跨层级长程残差连接的深度卷积神经网络。该模型受哺乳动物大脑组织启发，复现了皮层下区域到整个皮层层级的直接连接。实验表明，在多种架构（包括标准ResNet）中引入分层残差连接，可显著提升准确率并加速学习过程。对模型的深入分析揭示，其通过基于跳接提供的压缩表示来学习相对特征图，实现了分层组合性。该机制使网络能更高效地整合多层次语义信息。
原文摘要 · Abstract (English)
We present Hierarchical Residual Networks (HiResNets), deep convolutional neural networks with long-range residual connections between layers at different hierarchical levels. HiResNets draw inspiration on the organization of the mammalian brain by replicating the direct connections from subcortical areas to the entire cortical hierarchy. We show that the inclusion of hierarchical residuals in several architectures, including ResNets, results in a boost in accuracy and faster learning. A detailed analysis of our models reveals that they perform hierarchical compositionality by learning feature maps relative to the compressed representations provided by the skip connections.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。