通过时间复用提升物理神经网络深度，仅微调即可显著增强性能。
Time-multiplexed layer reuse for physical neural networks
- 利用时间分复用实现层间交替，突破物理网络深度限制。
- 在图像分类与自然语言任务中性能明显提升，接近传统网络效果。
- 适合硬件资源受限的物理神经网络研究者使用。
物理神经网络（PNN）是下一代计算的有力候选，但现有实现规模仍比现代数字神经网络小多个数量级，而后者的发展得益于参数量的快速增加。这类似于早期数字神经网络面临的困境，催生了参数复用的概念。本文聚焦于常见PNN原型中的关键瓶颈——权重缓慢重调校问题，提出时间索引深层交替层网络（TIDAL-Net），处于循环与深层网络之间的中间范式，专为当前PNN的规模与约束设计。TIDAL-Net利用许多PNN中存在的快前向动态与慢可训练权重/偏置的时间尺度分离特性，通过逐层时间复用增加有效深度，同时控制实现成本。在图像分类与自然语言处理任务上的数值实验表明，TIDAL-Net仅需对传统PNN进行微小修改，即可显著提升性能。
原文摘要 · Abstract (English)
Physical neural networks (PNNs) are promising candidates for next-generation computing, but existing demonstrations remain several orders of magnitude smaller than modern digital neural networks, whose recent advances have been driven by rapid growth in trainable parameters. This situation resembles the constraints of early digital neural networks, which led to ideas around parameter reuse. We investigate what similarly efficient hardware architectures may look like, focusing specifically on the common bottleneck of slow re-adjustment of the weights in PNNs. We propose the Time-Indexed Deep Alternating Layers Network (TIDAL-Net), which occupies an intermediate regime between recurrent and deep neural networks, specifically aimed at the scales and restrictions of common PNN prototypes. TIDAL-Net leverages the timescale separation found in many PNNs between fast forward dynamics and slowly trainable weights and biases, using layer-by-layer time multiplexing to increase effective depth while limiting implementation cost. Numerical experiments on image classification and natural language processing tasks show that TIDAL-Net improves performance with only minor modifications to conventional PNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。