提升视觉语言模型在未知测试分布下的泛化能力，通过多缓存原型学习实现更鲁棒的自适应。
Multi-Cache Enhanced Prototype Learning for Test-Time Generalization of Vision-Language Models
- 设计三缓存机制：熵缓存选低熵样本，对齐缓存融合模态信息，负样本缓存校准预测。
- 在15个下游任务上实现最优泛化性能，显著优于现有测试时自适应方法。
- 适合需要高鲁棒性的零样本场景，尤其在分布偏移下表现突出。
在零样本设置中，测试时自适应利用测试阶段的无标签数据调整预训练模型，以提升在未知测试分布上的性能。现有基于缓存的TTS方法依赖低熵准则选择样本构建原型，假设类内紧凑性。然而，在分布偏移下，低熵样本可能不可靠，导致原型无法保证类内紧凑。本研究发现缓存增强性能与类内紧凑性正相关。基于此，提出多缓存原型测试时自适应（MCP），包含三个缓存：熵缓存用于初始化原型表示，对齐缓存整合视觉与文本信息以实现紧凑类内分布，负样本缓存利用高熵样本进行预测校准。进一步提出MCP++框架，引入跨模态原型对齐与残差学习，实现原型残差微调。在15个下游任务上的对比与消融实验表明，所提方法与框架达到当前最优泛化性能。
原文摘要 · Abstract (English)
In zero-shot setting, test-time adaptation adjusts pre-trained models using unlabeled data from the test phase to enhance performance on unknown test distributions. Existing cache-enhanced TTA methods rely on a low-entropy criterion to select samples for prototype construction, assuming intra-class compactness. However, low-entropy samples may be unreliable under distribution shifts, and the resulting prototypes may not ensure compact intra-class distributions. This study identifies a positive correlation between cache-enhanced performance and intra-class compactness. Based on this observation, we propose a Multi-Cache enhanced Prototype-based Test-Time Adaptation (MCP) featuring three caches: an entropy cache for initializing prototype representations with low-entropy samples, an align cache for integrating visual and textual information to achieve compact intra-class distributions, and a negative cache for prediction calibration using high-entropy samples. We further developed MCP++, a framework incorporating cross-modal prototype alignment and residual learning, introducing prototype residual fine-tuning. Comparative and ablation experiments across 15 downstream tasks demonstrate that the proposed method and framework achieve state-of-the-art generalization performance. Project Page available at: https://zhaihaotian.github.io/MCP-ICCV25/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。