用加权最大均值差异缓解持续学习中的遗忘问题
Learning Dynamic Representations via An Optimally-Weighted Maximum Mean Discrepancy Optimization Framework for Continual Learning
- 通过多层级特征匹配机制，动态约束模型表示变化
- 在多个基准上达到当前最优性能，有效抑制灾难性遗忘
- 自适应正则化策略避免过度约束，适合长期学习场景
持续学习因其能够持续获取并保留知识而成为研究热点，但灾难性遗忘会严重损害模型性能。本文提出一种名为最优加权最大均值差异（OWMMD）的新框架，通过多层级特征匹配机制（MLFMM）对表示变化施加惩罚。同时引入自适应正则化优化（ARO）策略，自动评估各特征层的重要性，实现自适应权重调整。该方法可缓解过强正则化问题，提升后续任务学习能力。我们在多个基准上进行实验，结果表明所提方法性能优于现有基线，达到当前最优水平。
原文摘要 · Abstract (English)
Continual learning has emerged as a pivotal area of research, primarily due to its advantageous characteristic that allows models to persistently acquire and retain information. However, catastrophic forgetting can severely impair model performance. In this study, we address network forgetting by introducing a novel framework termed Optimally-Weighted Maximum Mean Discrepancy (OWMMD), which imposes penalties on representation alterations via a Multi-Level Feature Matching Mechanism (MLFMM). Furthermore, we propose an Adaptive Regularization Optimization (ARO) strategy to refine the adaptive weight vectors, which autonomously assess the significance of each feature layer throughout the optimization process, The proposed ARO approach can relieve the over-regularization problem and promote the future task learning. We conduct a comprehensive series of experiments, benchmarking our proposed method against several established baselines. The empirical findings indicate that our approach achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。