arXiv:2410.13563cs.LG2024-10被引 10

用随机噪声驱动学习,无需梯度也能自适应调整。

Ornstein-Uhlenbeck Adaptation as a Mechanism for Learning in Brains and Machines

  • 以奥恩斯坦-乌伦贝克过程模拟参数噪声,动态平衡探索与利用。
  • 在前馈和循环网络中实现监督与强化学习,支持元学习调参。
  • 适合类脑计算与神经形态硬件,或解释大脑中的噪声学习机制。

学习是智能系统的核心特性,存在于生物体与工程系统中。现代智能系统通常依赖梯度下降,但其对精确梯度和复杂信息流的需求,使得在生物与神经形态系统中实现困难。为此,我们提出一种新方法:利用系统参数中的噪声和全局强化信号。通过具有自适应动力学的奥恩斯坦-乌伦贝克过程(OUA),该方法在连续时间中基于误差预测偏差(类似奖励预测误差)调节学习,实现探索与利用的平衡。我们在多种任务中验证了该方法的有效性,涵盖前馈与递归系统的监督学习与强化学习,并证明其可实现元学习,自主调整超参数。结果表明,OUA为传统梯度方法提供了可行替代方案,适用于神经形态计算。同时,它也为大脑中由随机神经递质释放驱动的突触调整提供了可能机制。

原文摘要 · Abstract (English)

Learning is a fundamental property of intelligent systems, observed across biological organisms and engineered systems. While modern intelligent systems typically rely on gradient descent for learning, the need for exact gradients and complex information flow makes its implementation in biological and neuromorphic systems challenging. This has motivated the exploration of alternative learning mechanisms that can operate locally and do not rely on exact gradients. In this work, we introduce a novel approach that leverages noise in the parameters of the system and global reinforcement signals. Using an Ornstein-Uhlenbeck process with adaptive dynamics, our method balances exploration and exploitation during learning, driven by deviations from error predictions, akin to reward prediction error. Operating in continuous time, Orstein-Uhlenbeck adaptation (OUA) is proposed as a general mechanism for learning dynamic, time-evolving environments. We validate our approach across diverse tasks, including supervised learning and reinforcement learning in feedforward and recurrent systems. Additionally, we demonstrate that it can perform meta-learning, adjusting hyper-parameters autonomously. Our results indicate that OUA provides a viable alternative to traditional gradient-based methods, with potential applications in neuromorphic computing. It also hints at a possible mechanism for noise-driven learning in the brain, where stochastic neurotransmitter release may guide synaptic adjustments.

神经形态计算无梯度学习随机过程元学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。