利用文本特征提升跨域增量学习，不遗忘旧知识且无需标注任务身份。
CMAP: Cross-Modal Adaptive Prompting for Multi-Domain Task-Incremental Learning
- 用文本原型的余弦相似度实现任务路由，零参数开销且抗数据稀缺。
- 结合视觉与文本对齐得分，动态评估分类置信度，提升准确性。
- 适合需要高效更新、避免遗忘的多领域视觉任务场景。
多领域任务增量学习要求模型在不访问任务身份的情况下，依次学习视觉差异大的多个领域而不会遗忘旧知识。基于冻结视觉-语言模型的参数高效方法已取得显著进展，但现有方法仅依赖视觉特征进行任务路由、置信度估计和编码器适配，未利用CLIP的跨模态文本嵌入空间。本文提出三项贡献：1）文本空间任务路由，以冻结的CLIP文本原型的余弦相似度替代视觉高斯匹配，实现与顺序无关的路由，且在数据稀缺时仍鲁棒，零参数成本；2）多原型视觉-文本置信度，将单高斯类别建模替换为K均值视觉原型与跨模态对齐分数，在任务校准阈值下计算置信度；3）对称跨模态门控，将每层的Gumbel门扩展至文本编码器，根据批量图像特征调节，保持分布外输入下的跨模态对齐。在覆盖11个数据集、1201类的MTIL基准上，本方法在Order-I设置下达到74.2%迁移准确率、80.5%平均准确率、88.7%最终准确率，超越先前最优结果5.0、3.7、3.0个百分点，仅需250万可训练参数，无需外部数据。
原文摘要 · Abstract (English)
Multi-domain task-incremental learning requires a model to sequentially acquire knowledge across visually diverse domains without forgetting prior tasks, and without access to task identity at inference. Parameter-efficient methods built on frozen vision-language models have made strong progress, yet all existing approaches rely exclusively on visual features for task routing, confidence estimation, and encoder adaptation, leaving CLIP's cross-modal text embedding space entirely unexploited. We address this gap through three contributions. Text-space task routing replaces visual Gaussian matching with cosine similarity to frozen CLIP text prototypes, giving order-independent routing robust to data scarcity at zero parameter cost. Multi-prototype visual-textual confidence replaces single-Gaussian class modeling with K-means visual prototypes and cross-modal alignment scores under task-calibrated thresholds. Symmetric cross-modal gating extends per-layer Gumbel gates to the text encoder conditioned on batch image features, preserving cross-modal alignment on out-of-distribution inputs. On the MTIL benchmark spanning 11 datasets and 1201 classes, our method achieves 74.2% Transfer, 80.5% Average, and 88.7% Last under Order-I, surpassing the prior state of the art by 5.0, 3.7, and 3.0 percentage points with only 2.5M trainable parameters and no external data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。