用量子门控机制缓解预训练模型增量学习中的遗忘问题
Quantum-Gated Task-interaction Knowledge Distillation for Pre-trained Model-based Class-Incremental Learning
- 引入量子门控动态建模任务间关联，指导知识迁移
- 在多个数据集上实现当前最优的增量学习性能
- 适合需要持续学习且避免灾难性遗忘的研究场景
类增量学习（CIL）旨在从任务流中持续积累知识，并构建涵盖所有已见类别的统一分类器。尽管预训练模型（PTMs）在CIL中表现优异，但仍面临多任务子空间纠缠问题，当任务路由参数校准不佳或任务级表示固定僵化时，易导致灾难性遗忘。为此，我们提出一种新型量子门控任务交互知识蒸馏（QKD）框架，利用量子门控引导跨任务知识转移。具体地，引入量子门控任务调制机制，建模任务嵌入间的关联关系，动态捕捉样本与任务的相关性，适用于流式任务的联合训练与推理。基于量子门控输出，对旧任务到新适配器进行任务交互知识蒸馏，弥合独立任务子空间间的表征差距。大量实验表明，QKD能有效缓解遗忘，达到当前最优性能。
原文摘要 · Abstract (English)
Class-incremental learning (CIL) aims to continuously accumulate knowledge from a stream of tasks and construct a unified classifier over all seen classes. Although pretrained models (PTMs) have shown promising performance in CIL, they still struggle with the entanglement of multi-task subspaces, leading to catastrophic forgetting when task routing parameters are poorly calibrated or task-level representations are rigidly fixed. To address this issue, we propose a novel Quantum-Gated Task-interaction Knowledge Distillation (QKD) framework that leverages quantum gating to guide inter-task knowledge transfer. Specifically, we introduce a quantum-gated task modulation gating mechanism to model the relational dependencies among task embedding, dynamically capturing the sample-to-task relevance for both joint training and inference across streaming tasks. Guided by the quantum gating outputs, we perform task-interaction knowledge distillation guided by these task-embedding-level correlation weights from old to new adapters, enabling the model to bridge the representation gaps between independent task subspaces. Extensive experiments demonstrate that QKD effectively mitigates forgetting and achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。