用物理启发方法训练10层神经网络，图像识别误差仅比传统方法高1%。
Training a Predictive Coding Network on ImageNet using Equilibrium Propagation
- 将平衡传播算法改进后用于预测编码网络，实现大规模训练
- 在完整ImageNet上达到13.23%的顶级5分类错误率，接近12.2%的基准
- 首次在图像网规模展示预测编码网络与平衡传播的有效性
平衡传播（EP）是一种基于物理原理的训练框架，以往仅应用于小型问题。预测编码网络（PCNs）是另一类源于计算神经科学的能量模型，通常使用专用算法训练，尚未在大规模任务中验证。本文提出一种基于EP的PCN训练方法，结合中心化EP与新型平衡化方案，成功在全尺寸ImageNet上训练了10层卷积型PCN（VGG10），在顶-5分类任务中取得13.23%的测试错误率，接近12.2%的反向传播基准。这是首个在ImageNet尺度上实现的PCN与EP训练案例，显著提升了两者的可扩展性，表明其扩展瓶颈更可能来自系统计算特性而非框架本身限制。
原文摘要 · Abstract (English)
Equilibrium Propagation (EP) is a physics-based training framework that has primarily been employed in energy-based models, including continuous Hopfield networks, nonlinear resistive networks and coupled phase oscillators. However, EP's practical applications have so far remained limited to relatively small-scale problems. Predictive coding networks (PCNs), another class of energy-based models rooted in computational neuroscience, are typically trained with a specialized algorithm and have likewise not yet been demonstrated at large scale. In this work, we develop an EP-based training method for PCNs which combines the centered variant of EP with a novel equilibration scheme for PCNs. Using this approach, we train a 10-layer convolutional PCN (VGG10) on full-size ImageNet, achieving 13.23\% test error rate on the top-5 classification task, close to the 12.2\% backpropagation baseline. To our knowledge, this is the first demonstration of both PCNs and EP-based training at ImageNet scale. These results significantly extend the scalability of both approaches and suggest that the primary challenges in scaling EP in other physical systems may come more from the computational properties of these systems than from inherent limitations of the EP framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。