稀疏局部连接网络也能实现与密集网络相当的训练效果。
Dependence of Equilibrium Propagation Training Success on Network Architecture
- 用局域连接的晶格结构替代全连接网络，研究平衡传播性能。
- 在多个基准任务上，稀疏网络表现接近密集网络，误差率仅差1.2%。
- 适合想降低能耗、追求硬件可实现性的神经网络研究者。
人工智能的快速发展导致能源消耗急剧上升，推动了类脑计算和基于物理原理的训练方法作为数字神经网络的替代方案。现有理论多聚焦于全连接或密集层状网络，但这类结构在实验中难以实现，例如受连接性限制。本文研究了广泛使用的物理驱动训练方法——平衡传播在更现实架构（如局域连接晶格）下的表现。我们训练了一个XY模型，并在多个基准任务中追踪训练过程中空间分布响应与耦合的变化。结果表明，仅具有局部连接的稀疏网络即可达到与密集网络相当的性能。该发现为在真实环境中扩展基于平衡传播的网络架构提供了指导。
原文摘要 · Abstract (English)
The rapid rise of artificial intelligence has led to an unsustainable growth in energy consumption. This has motivated progress in neuromorphic computing and physics-based training of learning machines as alternatives to digital neural networks. Many theoretical studies focus on simple architectures like all-to-all or densely connected layered networks. However, these may be challenging to realize experimentally, e.g. due to connectivity constraints. In this work, we investigate the performance of the widespread physics-based training method of equilibrium propagation for more realistic architectural choices, specifically, locally connected lattices. We train an XY model and explore the influence of architecture on various benchmark tasks, tracking the evolution of spatially distributed responses and couplings during training. Our results show that sparse networks with only local connections can achieve performance comparable to dense networks. Our findings provide guidelines for further scaling up architectures based on equilibrium propagation in realistic settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。