提出新型网络结构，通过学习特征商提升深层网络性能
Quotient Network -- A Network Similar to ResNet but Learning Quotients
- 改学特征差为学习目标特征与现有特征的商
- 在多个数据集上超越ResNet且不增加参数量
- 设计新规则使网络可高效训练，适合改进现有深度模型
ResNet通过学习目标特征与现有特征的差异，实现了极深网络的有效训练。但该差异缺乏独立意义，且学习强度依赖绝对值而非相对比例，对现有特征规模敏感。本文提出商网络（Quotient Network），改而学习目标特征与现有特征的商，从根本上解决上述问题。为此设计了相应的网络构建规则，确保训练稳定并提升性能。在CIFAR10、CIFAR100和SVHN数据集上的实验表明，仅对ResNet进行微小调整即可实现显著性能提升，且无需额外参数，验证了该方法的有效性与实用性。
原文摘要 · Abstract (English)
The emergence of ResNet provides a powerful tool for training extremely deep networks. The core idea behind it is to change the learning goals of the network. It no longer learns new features from scratch but learns the difference between the target and existing features. However, the difference between the two kinds of features does not have an independent and clear meaning, and the amount of learning is based on the absolute rather than the relative difference, which is sensitive to the size of existing features. We propose a new network that perfectly solves these two problems while still having the advantages of ResNet. Specifically, it chooses to learn the quotient of the target features with the existing features, so we call it the quotient network. In order to enable this network to learn successfully and achieve higher performance, we propose some design rules for this network so that it can be trained efficiently and achieve better performance than ResNet. Experiments on the CIFAR10, CIFAR100, and SVHN datasets prove that this network can stably achieve considerable improvements over ResNet by simply making tiny corresponding changes to the original ResNet network without adding new parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。