arXiv:2412.08929cs.CV2024-12AAAI被引 10

通过循环聚合提示避免任务误判,缓解增量学习中的知识遗忘问题。

CAPrompt: Cyclic Prompt Aggregation for Pre-Trained Model Based Class Incremental Learning

  • 不依赖任务ID预测,用加权平均融合多个提示提升稳定性。
  • 在多个数据集上比现有方法提升2%-3%,显著减少性能下降。
  • 适合关注提示调优与持续学习的开发者和研究人员。

近期基于预训练模型的提示调优方法在类别增量学习(CIL)中表现优异,但通常需预测任务ID以选择对应提示,而任务预测错误会导致训练与推理时提示不一致,引发知识遗忘和性能下降。现有方法仅依赖预训练模型预测任务ID,未充分利用已学习的提示参数,导致预测效果不佳。为此,本文提出一种新型循环提示聚合方法(CAPrompt),通过循环聚合不同提示的知识,消除对任务ID预测的依赖。具体而言,在训练与推理阶段均采用创新的提示加权融合策略,利用多个提示的加权和克服提示不一致问题。理论分析表明,在凹性条件下,聚合提示的误差低于单个任务特定提示。因此,引入凹性约束与线性约束,确保提示学习满足该条件。此外,为更充分挖掘提示信息并实现更准确的权重分配,设计了循环权重预测机制:初始各任务权重相等,随后以循环方式自动调整至更优值。大量实验表明,所提方法在多个数据集上优于当前最优方法2%-3%。代码已开源。

原文摘要 · Abstract (English)

Recently, prompt tuning methods for pre-trained models have demonstrated promising performance in Class Incremental Learning (CIL). These methods typically involve learning task-specific prompts and predicting the task ID to select the appropriate prompts for inference. However, inaccurate task ID predictions can cause severe inconsistencies between the prompts used during training and inference, leading to knowledge forgetting and performance degradation. Additionally, existing prompt tuning methods rely solely on the pre-trained model to predict task IDs, without fully leveraging the knowledge embedded in the learned prompt parameters, resulting in inferior prediction performance. To address these issues, we propose a novel Cyclic Prompt Aggregation (CAPrompt) method that eliminates the dependency on task ID prediction by cyclically aggregating the knowledge from different prompts. Specifically, rather than predicting task IDs, we introduce an innovative prompt aggregation strategy during both training and inference to overcome prompt inconsistency by utilizing a weighted sum of different prompts. Thorough theoretical analysis demonstrates that under concave conditions, the aggregated prompt achieves lower error compared to selecting a single task-specific prompt. Consequently, we incorporate a concave constraint and a linear constraint to guide prompt learning, ensuring compliance with the concave condition requirement. Furthermore, to fully exploit the prompts and achieve more accurate prompt weights, we develop a cyclic weight prediction strategy. This strategy begins with equal weights for each task and automatically adjusts them to more appropriate values in a cyclical manner. Experiments on various datasets demonstrate that our proposed CAPrompt outperforms state-of-the-art methods by 2%-3%. Our code is available at https://github.com/zhoujiahuan1991/AAAI2025-CAPrompt.

提示调优增量学习知识遗忘循环聚合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。