用随机连接+可训练参数的半层结构，减少模型参数量并提升泛化能力。
Half-Layered Neural Networks
- 每层神经元仅保留两个可训练权重，其余权重随机固定。
- 在MNIST和FashionMNIST上以更少参数达到合理准确率。
- 适合追求轻量化模型或需要正则化的场景。
我们提出一种‘半层’隐藏单元结构：部分权重随机固定，部分可训练。每个半单元包含两阶段处理：第一阶段使用固定随机权重对输入加权求和；第二阶段通过两个可调权重进行缩放与平移，再经非线性激活。每个隐藏单元仅含两个可训练参数，不随输入维度变化。该结构可应用于深度网络的首层或任意中间层，甚至在卷积层后使用。在MNIST和FashionMNIST数据集上的实验表明，该方法利用随机连接的正则化效应，可在保持合理准确率的同时显著减少参数数量。
原文摘要 · Abstract (English)
We propose a ``half'' layer of hidden units that has some of its weights randomly set and some of them trained. A half unit is composed of two stages: First, it takes a weighted sum of its inputs with fixed random weights, and second, the total activation is multiplied and then translated using two modifiable weights, before the result is passed through a nonlinearity. The number of modifiable weights of each hidden unit is thus two and does not depend on the fan-in. We show how such half units can be used in the first or any later layer in a deep network, possibly following convolutional layers. Our experiments on MNIST and FashionMNIST data sets indicate the promise of half layers, where we can achieve reasonable accuracy with a reduced number of parameters due to the regularizing effect of the randomized connections.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。