用快速闭式解提升霍普菲尔德网络存储能力与抗噪性。
Kernel Ridge Regression for Efficient Learning of High-Capacity Hopfield Networks
- 采用核岭回归实现非迭代学习，通过闭式解快速求解双变量。
- 存储负载达1.5，抗噪性能接近最优的核逻辑回归方法。
- 训练速度比核逻辑回归快数个数量级，适合高容量记忆系统。
使用赫布学习的霍普菲尔德网络存在存储容量有限的问题。尽管监督方法如线性逻辑回归（LLR）有所改善，但核方法如核逻辑回归（KLR）显著提升了存储容量和抗噪能力，但需昂贵的迭代学习。本文提出核岭回归（KRR）作为高效替代方案，利用核技巧并通过回归预测双极状态，关键在于提供学习对偶变量的非迭代闭式解。我们在实验中对比了KRR与赫布、LLR和KLR的表现。结果表明，KRR实现了当前最优的存储容量（存储负载达1.5），且抗噪性接近KLR；更重要的是，其训练时间大幅缩减，比LLR快数个数量级，远超KLR，尤其在高存储负载下优势明显。该工作首次在霍普菲尔德网络学习中实证比较了KRR与KLR，确立了其作为高性能关联记忆构建方法的潜力，兼具媲美KLR的性能与显著的训练速度优势。
原文摘要 · Abstract (English)
Hopfield networks using Hebbian learning suffer from limited storage capacity. While supervised methods like Linear Logistic Regression (LLR) offer some improvement, kernel methods like Kernel Logistic Regression (KLR) significantly enhance storage capacity and noise robustness. However, KLR requires computationally expensive iterative learning. We propose Kernel Ridge Regression (KRR) as an efficient kernel-based alternative for learning high-capacity Hopfield networks. KRR utilizes the kernel trick and predicts bipolar states via regression, crucially offering a non-iterative, closed-form solution for learning dual variables. We evaluate KRR and compare its performance against Hebbian, LLR, and KLR. Our results demonstrate that KRR achieves state-of-the-art storage capacity (reaching a storage load of 1.5) and noise robustness, comparable to KLR. Crucially, KRR drastically reduces training time, being orders of magnitude faster than LLR and significantly faster than KLR, especially at higher storage loads. This establishes KRR as a potent and highly efficient method for building high-performance associative memories, providing comparable performance to KLR with substantial training speed advantages. This work provides the first empirical comparison between KRR and KLR in the context of Hopfield network learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。