arXiv:2607.09202cs.LGcs.AI2026-07

用干扰机制解释持续学习中的遗忘,提出无需重放的新方法

Interference and Retention in Continual Learning

论文配图:Interference and Retention in Continual Learning
图 1 · 摘自论文原文
  • 将遗忘视为任务间的干扰能量,直接建模而非事后补救
  • 在任务支持不重叠时可完全消除遗忘,重叠冲突时有最小不可逆损失
  • 新方法无需重放或Fisher信息,适合结构化任务流的持续学习

持续学习通常依赖事后机制如重放、弹性正则化或知识蒸馏。本文主张遗忘应直接建模为任务间的干扰。在固定特征情况下,学习新任务导致的遗忘正是旧任务受到的干扰能量。在深度网络中,该量可通过路径平均曲率以极少前向传播恢复。当任务支持集不重叠时,遗忘可结构性消除;当支持集重叠且方向冲突时,不可避免存在非零畸变下限。相同几何结构可最优地通过任务感知正交化合并模型。基于此分析,我们提出干扰门控功能分配(IGFA),一种无需重放、无Fisher信息的方法:任务对齐时共享方向,冲突时保护方向。在多个基准上,当任务结构可分时实现无损保留;当任务重叠时,将不可逆遗忘转化为可延迟恢复的可塑性。在差异任务流上达到最强无重放结构基线性能,在相似任务下优于无条件投影。

原文摘要 · Abstract (English)

Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation. This work argues that forgetting should instead be modeled directly as interference between tasks. In the frozen-feature regime, forgetting from learning a new task is exactly the interference energy induced on the old task. In deep networks, the same quantity is recovered through path-averaged curvature with minimal additional forward passes. When task supports are disjoint, forgetting can be eliminated structurally and when task supports overlap in conflicting directions, a non-zero distortion floor is unavoidable. The same geometry optimally merges models through task-aware orthogonalization. From this analysis we derive Interference-Gated Functional Allocation (IGFA), a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict. Across benchmarks, IGFA achieves lossless retention when tasks are structurally separable and moves unavoidable cost from irreversible forgetting into deferred but recoverable plasticity when they are not. It matches the strongest replay-free structural baselines on dissimilar-task streams and improves on unconditional projection when similarity makes transfer worth preserving.

持续学习干扰建模无重放任务对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。