不用键值对的持续学习新方法，提升稳定性和可扩展性
Key-Value Pair-Free Continual Learner via Task-Specific Prompt-Prototype
- 用任务专属提示与原型替代传统键值对
- 在多个数据集上优于主流提示方法，避免任务间干扰
- 适合研究持续学习与提示工程的学者参考
持续学习旨在让模型在保留旧知识的同时学习新知识。基于提示的方法在此领域表现优异，但通常依赖键值对结构，易引发任务间干扰并限制可扩展性。为此，本文提出一种新方法——任务特定提示原型（ProP），无需键值对。该方法通过任务专属提示增强当前任务特征学习，同时原型捕捉输入的代表性特征。推理时，通过将任务提示与对应原型结合生成预测。此外，在提示初始化阶段引入正则化约束，惩罚过大的数值以提升稳定性。在多个常用数据集上的实验表明，该方法有效克服了现有提示方法的局限性，摆脱对键值对的依赖，为未来持续学习研究提供了新思路。
原文摘要 · Abstract (English)
Continual learning aims to enable models to acquire new knowledge while retaining previously learned information. Prompt-based methods have shown remarkable performance in this domain; however, they typically rely on key-value pairing, which can introduce inter-task interference and hinder scalability. To overcome these limitations, we propose a novel approach employing task-specific Prompt-Prototype (ProP), thereby eliminating the need for key-value pairs. In our method, task-specific prompts facilitate more effective feature learning for the current task, while corresponding prototypes capture the representative features of the input. During inference, predictions are generated by binding each task-specific prompt with its associated prototype. Additionally, we introduce regularization constraints during prompt initialization to penalize excessively large values, thereby enhancing stability. Experiments on several widely used datasets demonstrate the effectiveness of the proposed method. In contrast to mainstream prompt-based approaches, our framework removes the dependency on key-value pairs, offering a fresh perspective for future continual learning research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。