学习是耗散过程,遗忘和正则化是系统自适应的必要条件。
Dissipative Learning: A Framework for Viable Adaptive Systems
- 用信息几何与热力学建模学习为受耗散约束的信念压缩演化。
- 证明费舍尔-罗信息正则化比欧氏正则化更优,耗散最小。
- 适用于持续学习和多智能体系统,强调生存性而非渐近最优。
我们提出一种观点:学习本质上是一个耗散过程。遗忘与正则化并非人为添加的技巧,而是自适应系统的结构性要求。结合信息论、热力学与信息几何,我们提出BEDS(贝叶斯涌现耗散结构)框架,将学习建模为在耗散约束下信念状态的压缩演化。核心贡献是条件最优定理,表明以信息散度衡量变化的费舍尔-罗正则化是唯一热力学最优的正则策略,实现最小耗散;而欧氏正则化被证明结构上次优。该框架统一了现有方法(岭回归、SIGReg、EMA、SAC)作为单一控制方程的特例。在此视角下,过拟合对应过度结晶,灾难性遗忘则反映耗散控制不足。框架区分了可结晶问题(信念收敛至稳定平衡)与可维持问题(需持续适应)。它自然扩展至持续学习与多智能体系统,其中生存性、适应下的稳定性及有限资源取代渐近最优成为首要标准。总体而言,本工作将学习重新定义为在耗散约束下维持可行信念状态的过程,为遗忘、正则化与稳定性提供了原则性视角。
原文摘要 · Abstract (English)
We propose a perspective in which learning is an intrinsically dissipative process. Forgetting and regularization are not heuristic add-ons but structural requirements for adaptive systems. Drawing on information theory, thermodynamics, and information geometry, we introduce the BEDS (Bayesian Emergent Dissipative Structures) framework, modeling learning as the evolution of compressed belief states under dissipation constraints. A central contribution is the Conditional Optimality Theorem, showing that Fisher-Rao regularization measuring change via information divergence rather than Euclidean distance is the unique thermodynamically optimal regularization strategy, achieving minimal dissipation. Euclidean regularization is shown to be structurally suboptimal. The framework unifies existing methods (Ridge, SIGReg, EMA, SAC) as special cases of a single governing equation. Within this view, overfitting corresponds to over-crystallization, while catastrophic forgetting reflects insufficient dissipation control. The framework distinguishes BEDS-crystallizable problems, where beliefs converge to stable equilibria, from BEDS-maintainable problems, which require continual adaptation. It extends naturally to continual and multi-agent systems, where viability, stability under adaptation and finite resources replaces asymptotic optimality as the primary criterion. Overall, this work reframes learning as maintaining viable belief states under dissipation constraints, providing a principled lens on forgetting, regularization, and stability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。