arXiv:2604.07108cs.LGcs.AI2026-04

用结构对齐理论重写持续学习,让模型自适应记忆而不遗忘。

Information as Structural Alignment: A Dynamical Theory of Continual Learning

  • 将信息视为结构对齐而非存储内容,通过动态调整实现记忆
  • 在CIFAR-100上接近零遗忘(BT = -0.004),棋类任务正向迁移+38.5 cp
  • 无需保存数据即可超越回放方法,适合研究长期学习机制者

灾难性遗忘并非工程缺陷,而是将知识作为全局参数叠加存储的数学结果。现有方法如正则化、回放和冻结子网络,均在共享参数基础上添加外部机制,无法从学习动力学中自然产生保留能力。本文提出信息积累框架(IBF),基于信息即结构对齐的假设,构建持续学习的新范式。其由运动定律驱动配置向高一致性演化,修改动力学则响应局部差异持续变形一致性景观。记忆、自主性与自我修正由此涌现,非外加模块。先在二维透明模型展示完整生命周期,再在三个领域验证:可控非平稳世界、独立评估的国际象棋(Stockfish)、Split-CIFAR-100(ViT编码器冻结)。所有场景下,IBF均超越回放方法且不存原始数据。在CIFAR-100上遗忘率极低(BT = -0.004),国际象棋实现+38.5 cp正向迁移,可控域遗忘减少43%。独立评估下,棋类表现平均领先+88.9 ± 2.8 cp,优于MLP与回放基线。

原文摘要 · Abstract (English)

Catastrophic forgetting is not an engineering failure. It is a mathematical consequence of storing knowledge as global parameter superposition. Existing methods, such as regularization, replay, and frozen subnetworks, add external mechanisms to a shared-parameter substrate. None derives retention from the learning dynamics themselves. This paper introduces the Informational Buildup Framework (IBF), an alternative substrate for continual learning, based on the premise that information is the achievement of structural alignment rather than stored content. In IBF, two equations govern the dynamics: a Law of Motion that drives configuration toward higher coherence, and Modification Dynamics that persistently deform the coherence landscape in response to localized discrepancies. Memory, agency, and self-correction arise from these dynamics rather than being added as separate modules. We first demonstrate the full lifecycle in a transparent two-dimensional toy model, then validate across three domains: a controlled non-stationary world, chess evaluated independently by Stockfish, and Split-CIFAR-100 with a frozen ViT encoder. Across all three, IBF achieves replay-superior retention without storing raw data. We observe near-zero forgetting on CIFAR-100 (BT = -0.004), positive backward transfer in chess (+38.5 cp), and 43% less forgetting than replay in the controlled domain. In chess, the framework achieves a mean behavioral advantage of +88.9 +/- 2.8 cp under independent evaluation, exceeding MLP and replay baselines.

持续学习结构对齐自纠正国际象棋

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。