用少量数据实现持续学习，让模型不断适应新领域。
Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidation
- 通过视觉与语言原型融合，保留基础域潜在空间以支持后续学习。
- 在少样本场景下，性能比现有方法最高提升16%。
- 适合需要持续适应新数据领域的研究者使用。
现有领域增量学习（DIL）方法需要大量数据来适应新领域，且在数据稀缺时易过拟合。本文提出一个较少被关注的问题——少样本领域增量学习（FSDIL），应对极端数据短缺的挑战。为此，提出一种新算法：持续视觉-语言整合（CVLC），核心思想是利用基础域的潜在空间预留，并结合双协同投影（DCP）实现参数高效微调。首先，通过大语言模型生成多模板和同义词，引导语言原型生成，再与视觉原型融合；随后，借助DCP技术持续适应不断到来的新领域，通过基础域的潜在空间预留使模型具备对未见领域的泛化能力。模型采用共享与领域专属组件相结合的结构，兼顾通用知识与领域细节。在多个基准测试中，该方法表现优于已有方法，最高提升达16%。代码已公开于 https://github.com/Naeem-Paeedeh/CVLC。
原文摘要 · Abstract (English)
Existing domain-incremental learning (DIL) strategies call for massive amounts of data to adapt to new domains and suffer from the overfitting problem in the case of data scarcity. This paper puts forward a relatively uncharted problem, namely, few-shot domain incremental learning (FSDIL), taking into account the problem of extreme data shortages in the realm of DIL. A novel algorithm, namely Continual Vision-Language Consolidation (CVLC), is proposed to address the FSDIL problem, where the key idea lies in the concept of latent space reservation in the base domain coupled with dual coalescent projection (DCP) as a parameter-efficient fine-tuning method. First, the vision prototype is calibrated while multiple templates and synonyms are generated via LLMs to induce the language prototype. The vision and language prototypes are fused. Adaptation to never-ending arrivals of new domains is done by the DCP technique, fine-tuned in such a way to prepare the model to unseen domains via latent-space reservations committed in the base domain. CVLC is structured under shared and domain-specific components to combine general knowledge and domain-specific details. The advantage of our approach is demonstrated through a range of benchmark problems and comparisons with prior arts, in which CVLC outperforms them by up to a 16% gap. Our codes are shared publicly in https://github.com/Naeem-Paeedeh/CVLC .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。