让视觉语言模型在测试时自动优化提示，应对域偏移问题。
MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models
- 通过元学习自监督任务动态生成样本增强，提升提示调优表达力。
- 在跨数据集和域泛化任务上达到当前最佳性能，显著提升鲁棒性。
- 适合需要强泛化能力的视觉语言模型部署场景。
视觉语言模型（如CLIP）虽具备出色的零样本泛化能力，但在测试阶段面对域偏移仍敏感。测试时提示调优（TPT）通过固定增强方式调整提示，但在更复杂场景下表现有限。本文提出元测试时提示调优（MetaTPT），一种基于元学习的框架，通过学习自监督辅助任务来指导测试时提示调优。该任务能为每个样本动态学习参数化增强，实现更丰富的特征变换。MetaTPT采用双循环优化：内层学习生成信息丰富视图的自监督任务，外层通过视图间一致性约束进行提示调优。通过将增强学习与提示调优耦合，显著提升了在域偏移下的测试时适应能力。大量实验表明，MetaTPT在域泛化与跨数据集基准上均达到当前最优表现。
原文摘要 · Abstract (English)
Vision-language models (VLMs) such as CLIP exhibit strong zero-shot generalization but remain sensitive to domain shifts at test time. Test-time prompt tuning (TPT) mitigates this issue by adapting prompts with fixed augmentations, which may falter in more challenging settings. In this work, we propose Meta Test-Time Prompt Tuning (MetaTPT), a meta-learning framework that learns a self-supervised auxiliary task to guide test-time prompt tuning. The auxiliary task dynamically learns parameterized augmentations for each sample, enabling more expressive transformations that capture essential features in target domains. MetaTPT adopts a dual-loop optimization paradigm: an inner loop learns a self-supervised task that generates informative views, while the outer loop performs prompt tuning by enforcing consistency across these views. By coupling augmentation learning with prompt tuning, MetaTPT improves test-time adaptation under domain shifts. Extensive experiments demonstrate that MetaTPT achieves state-of-the-art performance on domain generalization and cross-dataset benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。