arXiv:2510.23756cs.LGcs.AI2025-10

提出概念形成机制缓解持续学习中的灾难性遗忘问题。

Explaining Robustness to Catastrophic Forgetting Through Incremental Concept Formation

  • 基于信息论的分层概念形成,动态调整模型结构以保留旧知识。
  • 稀疏更新与自适应重组织使模型在多个数据集上保持稳定表现。
  • 适合研究持续学习、认知计算或对可解释性有要求的场景。

灾难性遗忘仍是持续学习的核心挑战,即模型在不断学习新知识时会丢失已有知识。此前我们提出Cobweb/4V——一种分层概念形成模型,在视觉任务中展现出对遗忘的良好鲁棒性。本文针对其稳定性提出三个假设:(1) 自适应结构重组织增强知识保留;(2) 稀疏且选择性更新减少干扰;(3) 基于充分统计量的信息论学习优于梯度反向传播。通过对比Cobweb/4V与神经基线(包括本文提出的神经实现CobwebNN),在不同复杂度数据集(MNIST、Fashion-MNIST、MedMNIST、CIFAR-10)上的实验表明:自适应重构提升学习灵活性,稀疏更新减轻干扰,信息论学习无需回看旧数据即可保持先验知识。这些发现揭示了缓解遗忘的关键机制,凸显了基于概念与信息论的方法在构建稳定、自适应持续学习系统中的潜力。

原文摘要 · Abstract (English)

Catastrophic forgetting remains a central challenge in continual learning, where models are required to integrate new knowledge over time without losing what they have previously learned. In prior work, we introduced Cobweb/4V, a hierarchical concept formation model that exhibited robustness to catastrophic forgetting in visual domains. Motivated by this robustness, we examine three hypotheses regarding the factors that contribute to such stability: (1) adaptive structural reorganization enhances knowledge retention, (2) sparse and selective updates reduce interference, and (3) information-theoretic learning based on sufficiency statistics provides advantages over gradient-based backpropagation. To test these hypotheses, we compare Cobweb/4V with neural baselines, including CobwebNN, a neural implementation of the Cobweb framework introduced in this work. Experiments on datasets of varying complexity (MNIST, Fashion-MNIST, MedMNIST, and CIFAR-10) show that adaptive restructuring enhances learning plasticity, sparse updates help mitigate interference, and the information-theoretic learning process preserves prior knowledge without revisiting past data. Together, these findings provide insight into mechanisms that can mitigate catastrophic forgetting and highlight the potential of concept-based, information-theoretic approaches for building stable and adaptive continual learning systems.

持续学习概念形成信息论遗忘缓解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。