用电阻二极管电路实现深度神经网络，可直接硬件训练
Circuit realization and hardware linearization of monotone operator equilibrium networks
- 用电阻二极管网络模拟深层ReLU神经网络的求解过程
- 硬件中直接计算梯度，支持端到端电路训练
- 可扩展至多层结构，适配非对称网络设计
本文证明，电阻-二极管网络的端口行为等价于一个在无限深度极限下的ReLU单调算子均衡网络（monotone operator equilibrium network）的解，从而在模拟硬件中实现了神经网络的简洁构造。我们进一步提出一种称为硬件线性化（hardware linearization）的方法，可在硬件层面直接计算该电路的梯度，使网络具备在硬件中直接训练的能力，已在器件级电路仿真中验证。研究还拓展至多级电阻-二极管网络级联，可用于实现前馈及其他非对称网络结构。最后，发现不同非线性元件会引出不同的激活函数，并基于非理想二极管模型提出新型“二极管ReLU”激活函数。
原文摘要 · Abstract (English)
It is shown that the port behavior of a resistor-diode network corresponds to the solution of a ReLU monotone operator equilibrium network (a neural network in the limit of infinite depth), giving a parsimonious construction of a neural network in analog hardware. We furthermore show that the gradient of such a circuit can be computed directly in hardware, using a procedure we call hardware linearization. This allows the network to be trained in hardware, which we demonstrate with a device-level circuit simulation. We extend the results to cascades of resistor-diode networks, which can be used to implement feedforward and other asymmetric networks. We finally show that different nonlinear elements give rise to different activation functions, and introduce the novel diode ReLU which is induced by a non-ideal diode model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。