提出新方法缓解持续学习中遗忘问题,无需存储旧数据。
Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning
- 用残差聚合与稀疏提示选择提升表示适应能力
- 在三个无任务标签的基准上达到最高平均准确率
- 适合资源受限场景下的持续学习应用
持续学习(CL)面临灾难性遗忘问题,尤其在无任务标识的领域增量学习(DIL)中更为严重。现有基于提示的持续学习(PCL)方法仅靠提示调整效果有限,因提示选择不佳且分类器在领域漂移下不稳定。本文提出Residual SODAP,联合实现基于提示的表征适应与分类器级知识保留。方法融合α-entmax稀疏提示选择、残差聚合、无数据知识蒸馏与伪特征回放、基于提示使用量的漂移检测,以及不确定性感知的多损失平衡。在三个不依赖任务标识且不需存储历史数据的DIL基准上,该方法分别取得0.850/0.047(DR)、0.760/0.031(Skin Cancer)和0.995/0.003(CORe50)的平均准确率与平均遗忘率,达到当前最优性能。
原文摘要 · Abstract (English)
Continual learning (CL) suffers from catastrophic forgetting, which is exacerbated in domain-incremental learning (DIL) where task identifiers are unavailable and storing past data is infeasible. While prompt-based CL (PCL) adapts representations with a frozen backbone, we observe that prompt-only improvements are often insufficient due to suboptimal prompt selection and classifier-level instability under domain shifts. We propose Residual SODAP, which jointly performs prompt-based representation adaptation and classifier-level knowledge preservation. Our framework combines $α$-entmax sparse prompt selection with residual aggregation, data-free distillation with pseudo-feature replay, prompt-usage--based drift detection, and uncertainty-aware multi-loss balancing. Across three DIL benchmarks without task IDs or extra data storage, Residual SODAP achieves state-of-the-art AvgACC/AvgF of 0.850/0.047 (DR), 0.760/0.031 (Skin Cancer), and 0.995/0.003 (CORe50).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。