发现持续学习中遗忘的知识仍可稳定恢复，关键在几何结构而非信息丢失。
The Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual Learning
- 提出恢复子空间维数k_t，量化恢复所需最小方向数
- 平均k_t=8.0，尽管表征漂移显著，恢复维度仍稳定
- 表征角度漂移强预测恢复能力，模型解释82.2%方差
灾难性遗忘常被视为顺序学习中先前知识的破坏。基于可访问性坍缩框架，我们研究了持续学习中可恢复性的几何结构。通过在Split CIFAR-100上对顺序训练的ResNet-18进行分析，考察了十项任务中的可恢复性、表征漂移与恢复复杂性。引入恢复子空间维数（k_t），衡量保留90%完整探测性能所需的最少奇异方向数。与恢复扩散假说相反，尽管存在显著表征漂移，恢复维度在整个训练过程中保持稳定（均值k_t=8.0）。主角度漂移强烈预测可恢复性（r = -0.862），且一个简单几何模型解释了82.2%的可恢复性方差。这些发现支持稳定恢复流形假说，表明遗忘知识仍以紧凑形式可解码，尽管表征发生重组。结果表明，灾难性遗忘主要是可访问性与流形对齐问题，而非信息破坏。
原文摘要 · Abstract (English)
Catastrophic forgetting is often viewed as the destruction of previously learned knowledge during sequential learning. Building on the Accessibility Collapse framework, we investigate the geometric structure of recoverability in continual learning. Using Split CIFAR-100 and a sequentially trained ResNet-18, we analyze recoverability, representational drift, and recovery complexity across ten tasks. We introduce Recovery Subspace Dimensionality (k_t), a measure of the minimum number of singular directions required to preserve 90 percent of full probe performance. Contrary to our Recoverability Diffusion hypothesis, recovery dimensionality remains stable throughout training (mean k_t = 8.0) despite substantial representational drift. Principal-angle drift strongly predicts recoverability (r = -0.862), and a simple geometric model explains 82.2 percent of recoverability variance. These findings support the Stable Recovery Manifold hypothesis, suggesting that forgotten knowledge remains compactly decodable despite representational reorganization. The results indicate that catastrophic forgetting is primarily an accessibility and manifold-alignment problem rather than information destruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。