用可学习的领域专属提示,动态分配并调整模型以应对持续变化的数据分布。
Dynamic Prompt Allocation and Tuning for Continual Test-Time Adaptation
- 引入领域特定提示,分治不同数据域的参数更新。
- 在未知领域时自动分配新提示,已知则直接调用对应提示。
- 通过互信息最大化与结构正则化实现高效微调,适合持续学习场景。
持续测试时适应(CTTA)旨在将预训练模型适配到不断演化的目标分布,以应对真实环境的动态性。现有方法通常通过显式正则化约束参数变化以缓解灾难性遗忘,但因使用单一共享模型适应所有领域,仍导致严重的域间干扰。本文提出可学习的领域特定提示,引导模型针对不同目标域进行适配,部分解耦各域参数空间。当目标样本无领域标识时,提出动态提示分配与调优(PAINT)方法,利用查询机制判断样本是否来自已知域或新域:已知域直接选取对应提示,新域则分配新提示;随后通过互信息最大化与结构正则化进行提示微调。在三个基准数据集上的大量实验表明,PAINT 方法在 CTTA 中表现优异。代码已开源:https://github.com/Cadezzyr/PAINT。
原文摘要 · Abstract (English)
Continual test-time adaptation (CTTA) has recently emerged to adapt a pre-trained source model to continuously evolving target distributions, which accommodates the dynamic nature of real-world environments. To mitigate the risk of catastrophic forgetting in CTTA, existing methods typically incorporate explicit regularization terms to constrain the variation of model parameters. However, they cannot fundamentally resolve catastrophic forgetting because they rely on a single shared model to adapt across all target domains, which inevitably leads to severe inter-domain interference. In this paper, we introduce learnable domain-specific prompts that guide the model to adapt to corresponding target domains, thereby partially disentangling the parameter space of different domains. In the absence of domain identity for target samples, we propose a novel dynamic Prompt AllocatIon aNd Tuning (PAINT) method, which utilizes a query mechanism to dynamically determine whether the current samples come from a known domain or an unexplored one. For known domains, the corresponding domain-specific prompt is directly selected, while for previously unseen domains, a new prompt is allocated. Prompt tuning is subsequently performed using mutual information maximization along with structural regularization. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our PAINT method for CTTA. We have released our code at https://github.com/Cadezzyr/PAINT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。