让提示词动态进化，提升持续学习的多样性与效果
RainbowPrompt: Diversity-Enhanced Prompt-Evolving for Continual Learning
- 设计可自适应聚合任务专属提示的演化机制
- 在图像和视频任务上分别提升9.07%和7.40%准确率
- 适合需要持续学习且无重放场景的应用
基于提示的持续学习通过微调少量参数实现无需重放的学习，同时保持预训练模型冻结。为应对序列任务的复杂需求，需有效整合任务特定知识到提示中。然而，现有方法依赖固定提示或纠缠共享空间生成提示，限制了提示表征的多样性。为此，我们提出一种新颖的提示演化机制，自适应地将基础提示（即任务特定提示）整合为统一提示，并确保多样性。通过变换与对齐已有及新引入的提示，本方法持续演化累积知识以支持新任务学习。此外，引入可学习的概率门控机制，动态决定演化过程中激活的层数。我们在类增量学习的图像分类与视频动作识别任务上验证该方法，所有场景下平均性能分别优于现有方法9.07%和7.40%。
原文摘要 · Abstract (English)
Prompt-based continual learning provides a rehearsal-free solution by tuning small sets of parameters while keeping pre-trained models frozen. To meet the complex demands of sequential tasks, it is crucial to integrate task-specific knowledge within prompts effectively. However, existing works rely on either fixed learned prompts (i.e., prompts whose representations remain unchanged during new task learning) or on prompts generated from an entangled task-shared space, limiting the representational diversity of the integrated prompt. To address this issue, we propose a novel prompt-evolving mechanism to adaptively aggregate base prompts (i.e., task-specific prompts) into a unified prompt while ensuring diversity. By transforming and aligning base prompts, both previously learned and newly introduced, our approach continuously evolves accumulated knowledge to facilitate learning new tasks. We further introduce a learnable probabilistic gate that adaptively determines which layers to activate during the evolution process. We validate our method on image classification and video action recognition tasks in class-incremental learning, achieving average gains of 9.07% and 7.40% over existing methods across all scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。