受人类学习启发,提出前向-协作-后向学习框架提升神经网络性能
Forward-Cooperation-Backward (FCB) learning in a Multi-Encoding Uni-Decoding neural network architecture
- 设计多编码单解码架构,引入横向连接实现同伴协作式学习
- 在4个数据集上实现高效降维,保留数据细粒度特征
- 适合需要可解释性与协同优化的机器学习任务
主流神经网络训练方法是反向传播,近年也出现了前向-前向学习技术。然而,人类学习并非单一依赖其中任何一种方式,而是融合了前向探索、反向修正与同伴协作。人们首先自主理解新概念,并通过层级迭代优化;遇到困惑时,常通过与同伴讨论(即协作)来解决;即使如此,仍可能存疑,需通过对比原始文献持续修正。受此启发,本文提出一种模仿人类学习机制的前向-协作-后向(FCB)学习策略,并设计了多编码单解码神经网络架构。该架构引入特殊横向突触连接以实现协作。在四个流行数据集上验证了其降维性能,测试表明其能有效保留低秩嵌入中的数据细粒度特性。下游分类任务亦表现良好。通过收敛性分析进一步验证了FCB策略的有效性。
原文摘要 · Abstract (English)
The most popular technique to train a neural network is backpropagation. Recently, the Forward-Forward technique has also been introduced for certain learning tasks. However, in real life, human learning does not follow any of these techniques exclusively. The way a human learns is basically a combination of forward learning, backward propagation and cooperation. Humans start learning a new concept by themselves and try to refine their understanding hierarchically during which they might come across several doubts. The most common approach to doubt solving is a discussion with peers, which can be called cooperation. Cooperation/discussion/knowledge sharing among peers is one of the most important steps of learning that humans follow. However, there might still be a few doubts even after the discussion. Then the difference between the understanding of the concept and the original literature is identified and minimized over several revisions. Inspired by this, the paper introduces Forward-Cooperation-Backward (FCB) learning in a deep neural network framework mimicking the human nature of learning a new concept. A novel deep neural network architecture, called Multi Encoding Uni Decoding neural network model, has been designed which learns using the notion of FCB. A special lateral synaptic connection has also been introduced to realize cooperation. The models have been justified in terms of their performance in dimension reduction on four popular datasets. The ability to preserve the granular properties of data in low-rank embedding has been tested to justify the quality of dimension reduction. For downstream analyses, classification has also been performed. An experimental study on convergence analysis has been performed to establish the efficacy of the FCB learning strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。