证明对比学习在电阻网络中可收敛，为类脑计算提供理论支持
Convergence of energy-based learning in linear resistive networks
- 将对比学习等价为带凸约束的梯度下降
- 证明了在特定步长下算法必然收敛
- 结果扩展到随机版本，适用于实际硬件实现
能量基学习算法是反向传播的替代方案，适合在模拟电子设备中分布式实现。然而，其收敛性缺乏严格理论。本文首次对线性可调电阻网络中的对比学习算法进行分析，发现该算法等价于在具有Lipschitz连续梯度的凸函数上进行投影梯度下降，从而在一定步长范围内保证算法收敛。这一结论进一步推广至对比学习的随机变体，为类脑计算硬件中的学习机制提供了坚实的理论基础。
原文摘要 · Abstract (English)
Energy-based learning algorithms are alternatives to backpropagation and are well-suited to distributed implementations in analog electronic devices. However, a rigorous theory of convergence is lacking. We make a first step in this direction by analysing a particular energybased learning algorithm, Contrastive Learning, applied to a network of linear adjustable resistors. It is shown that, in this setup, Contrastive Learning is equivalent to projected gradient descent on a convex function with Lipschitz continuous gradient, giving a guarantee of convergence of the algorithm for a range of stepsizes. This convergence result is then extended to a stochastic variant of Contrastive Learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。