首次量化分析持续学习中虚假遗忘的深层原因,提出实时检测与修复方案。
Real Time Detection and Quantitative Analysis of Spurious Forgetting in Continual Learning
- 提出浅层与深层对齐框架,量化评估模型输出各位置的对齐程度。
- 发现现有方法仅保持前3-5个输出词的对齐,导致易发生可逆的虚假遗忘。
- 支持实时检测与自适应恢复,适用于大模型持续学习场景。
灾难性遗忘仍是大语言模型持续学习中的核心挑战。近期研究指出,性能下降可能源于任务对齐破坏引发的虚假遗忘,而非真实知识丢失。然而,这些工作仅定性描述对齐机制,依赖事后分析,缺乏自动区分能力。本文提出浅层与深层对齐框架,首次实现对齐深度的量化刻画。我们发现当前任务对齐方法存在浅层对齐——仅在前3-5个输出词上维持对齐,使模型易受遗忘影响。这解释了虚假遗忘的发生机制、可逆性及微调攻击的有效性。为此,我们构建完整框架:(1) 提出0-1尺度的量化指标,测量各输出位置的对齐深度;(2) 实时检测训练中浅层对齐现象;(3) 提供可视化与恢复预测分析工具;(4) 设计自适应缓解策略,自动区分遗忘类型并促进深层对齐。在多个数据集与模型架构(Qwen2.5-3B至Qwen2.5-32B)上的实验表明,识别准确率达86.2%-90.6%,推动深层对齐可使模型抗遗忘能力提升3.3%-7.1%。
原文摘要 · Abstract (English)
Catastrophic forgetting remains a fundamental challenge in continual learning for large language models. Recent work revealed that performance degradation may stem from spurious forgetting caused by task alignment disruption rather than true knowledge loss. However, this work only qualitatively describes alignment, relies on post-hoc analysis, and lacks automatic distinction mechanisms. We introduce the shallow versus deep alignment framework, providing the first quantitative characterization of alignment depth. We identify that current task alignment approaches suffer from shallow alignment - maintained only over the first few output tokens (approximately 3-5) - making models vulnerable to forgetting. This explains why spurious forgetting occurs, why it is reversible, and why fine-tuning attacks are effective. We propose a comprehensive framework addressing all gaps: (1) quantitative metrics (0-1 scale) to measure alignment depth across token positions; (2) real-time detection methods for identifying shallow alignment during training; (3) specialized analysis tools for visualization and recovery prediction; and (4) adaptive mitigation strategies that automatically distinguish forgetting types and promote deep alignment. Extensive experiments on multiple datasets and model architectures (Qwen2.5-3B to Qwen2.5-32B) demonstrate 86.2-90.6% identification accuracy and show that promoting deep alignment improves robustness against forgetting by 3.3-7.1% over baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。