arXiv:2604.01116cs.CV2026-04

用视觉原型指导文本提示选择,缓解持续学习中的遗忘问题。

ProTPS: Prototype-Guided Text Prompt Selection for Continual Learning

  • 基于类别视觉原型筛选并学习独特文本提示。
  • 在三种设置下性能接近理论上限,优于最新方法。
  • 适用于真实世界长尾分布场景,适合持续学习研究者。

持续学习中,基于文本提示的方法利用文本编码器和可学习提示来编码随时间变化的类别语义特征。现有方法常面临如何学习唯一提示的问题,这些提示隐含新类别的语义信息,避免与已训练类别的特征重叠,从而缓解灾难性遗忘。为此,本文提出原型引导的文本提示选择(ProTPS),通过增强训练灵活性以促进独特提示的学习。具体而言,ProTPS学习类别特定的视觉原型和文本提示,视觉原型用于指导每个类别的提示选择与学习。我们在类别增量(CI)和跨数据集持续学习(CDC)设置下评估了ProTPS。由于其性能接近上界,我们进一步构建了一个包含112种海洋生物的真实世界数据集Marine112,覆盖六年时间跨度,具有自然长尾分布,适用于类别与领域增量(CDI)学习。三种设置下的实验结果表明,ProTPS显著优于近期先进方法。代码与数据集将在论文接受后公开。

原文摘要 · Abstract (English)

For continual learning, text-prompt-based methods leverage text encoders and learnable prompts to encode semantic features for sequentially arrived classes over time. A common challenge encountered by existing works is how to learn unique text prompts, which implicitly carry semantic information of new classes, so that the semantic features of newly arrived classes do not overlap with those of trained classes, thereby mitigating the catastrophic forgetting problem. To address this challenge, we propose a novel approach Prototype-guided Text Prompt Selection (ProTPS)'' to intentionally increase the training flexibility thus encouraging the learning of unique text prompts. Specifically, our ProTPS learns class-specific vision prototypes and text prompts. Vision prototypes guide the selection and learning of text prompts for each class. We first evaluate our ProTPS in both class incremental (CI) setting and cross-datasets continual (CDC) learning setting. Because our ProTPS achieves performance close to the upper bounds, we further collect a real-world dataset with 112 marine species collected over a span of six years, named Marine112, to bring new challenges to the community. Marine112 is authentically suited for the class and domain incremental (CDI) learning setting and is under natural long-tail distribution. The results under three settings show that our ProTPS performs favorably against the recent state-of-the-art methods. The implementation code and Marine112 dataset will be released upon the acceptance of our paper.

持续学习文本提示原型学习长尾分布

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。