提出多查询多键匹配机制，精准选择持续学习中的提示词。
Multiple Queries with Multiple Keys: A Precise Prompt Matching Paradigm for Prompt-based Continual Learning
- 用多查询与多键设计实现细粒度的提示匹配，提升搜索精度。
- 在挑战性场景下提示匹配率提升超30%，性能达当前最优。
- 适合需要高精度提示选择的持续学习任务，如动态环境下的模型更新。
持续学习要求模型在动态环境中不断获取新知识，同时避免遗忘旧知识。基于提示的持续学习方法通过提示扩展与选择有效缓解灾难性遗忘问题。然而，现有方法常因提示选择准确率低，导致模型接收有偏知识并产生偏差预测。为此，本文提出多查询多键（MQMK）提示匹配范式，旨在选择训练数据分布最接近测试样本的提示。具体而言，多查询通过引入任务特定知识实现精准广度搜索；多键则通过细粒度表征训练样本特征分布，实现深度搜索。每个查询针对特定任务进行局部匹配，减少跨查询干扰。实验表明，MQMK在挑战性场景下提示匹配率提升超过30%，并在三个主流持续学习基准上达到先进性能。代码已开源：https://github.com/DunweiTu/MQMK。
原文摘要 · Abstract (English)
Continual learning requires machine learning models to continuously acquire new knowledge in dynamic environments while avoiding the forgetting of previous knowledge. Prompt-based continual learning methods effectively address the issue of catastrophic forgetting through prompt expansion and selection. However, existing approaches often suffer from low accuracy in prompt selection, which can result in the model receiving biased knowledge and making biased predictions. To address this issue, we propose the Multiple Queries with Multiple Keys (MQMK) prompt matching paradigm for precise prompt selection. The goal of MQMK is to select the prompts whose training data distribution most closely matches that of the test sample. Specifically, Multiple Queries enable precise breadth search by introducing task-specific knowledge, while Multiple Keys perform deep search by representing the feature distribution of training samples at a fine-grained level. Each query is designed to perform local matching with a designated task to reduce interference across queries. Experiments show that MQMK enhances the prompt matching rate by over 30\% in challenging scenarios and achieves state-of-the-art performance on three widely adopted continual learning benchmarks. The code is available at https://github.com/DunweiTu/MQMK.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。