让神经网络在混沌边缘学习，应对数据突变时更稳定。
Lyapunov Learning at the Onset of Chaos
- 将神经网络置于混沌边缘，利用李雅普诺夫指数控制适应性。
- 在洛伦兹系统参数突变场景下，性能提升约96%。
- 适合需要持续学习、应对突发变化的动态系统应用。
深度学习系统在处理制度转换和非平稳时间序列时面临重大挑战。在线学习中，新信息的引入可能破坏已有数据并改变模型整体范式，尤其在非平稳数据源下。因此，神经系统需快速适应新范式，同时保留对整体问题至关重要的历史知识。本文提出一种名为‘李雅普诺夫学习’的新训练算法，利用非线性混沌动力系统的特性，为潜在制度转换做好准备。受斯图尔特·考夫曼‘邻近可能’理论启发，该方法利用解空间中的局部未探索区域实现灵活适应。神经网络被设计为在混沌边缘运行，其最大李雅普诺夫指数随时间趋近于零。实验表明，该方法在非平稳系统制度转换中表现显著优于常规训练，在洛伦兹混沌系统参数突变场景下，损失比提升约96%。
原文摘要 · Abstract (English)
Handling regime shifts and non-stationary time series in deep learning systems presents a significant challenge. In the case of online learning, when new information is introduced, it can disrupt previously stored data and alter the model's overall paradigm, especially with non-stationary data sources. Therefore, it is crucial for neural systems to quickly adapt to new paradigms while preserving essential past knowledge relevant to the overall problem. In this paper, we propose a novel training algorithm for neural networks called \textit{Lyapunov Learning}. This approach leverages the properties of nonlinear chaotic dynamical systems to prepare the model for potential regime shifts. Drawing inspiration from Stuart Kauffman's Adjacent Possible theory, we leverage local unexplored regions of the solution space to enable flexible adaptation. The neural network is designed to operate at the edge of chaos, where the maximum Lyapunov exponent, indicative of a system's sensitivity to small perturbations, evolves around zero over time. Our approach demonstrates effective and significant improvements in experiments involving regime shifts in non-stationary systems. In particular, we train a neural network to deal with an abrupt change in Lorenz's chaotic system parameters. The neural network equipped with Lyapunov learning significantly outperforms the regular training, increasing the loss ratio by about $96\%$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。