让所有测试样本都发挥作用，通过均值迁移提升视觉语言模型的泛化能力。
Unleashing the Potential of All Test Samples: Mean-Shift Guided Test-Time Adaptation
- 利用kNN均值迁移统一优化所有测试样本特征，突破原始特征空间限制。
- 在多个分布外和跨数据集测试中性能超越现有方法，显著提升分类稳定性。
- 无需额外训练，适合部署于资源受限的实时推理场景。
视觉语言模型（如CLIP）具备强大泛化能力，但在测试时遭遇分布偏移表现下降。现有无训练测试时自适应（TTA）方法仅在CLIP原始特征空间内操作，依赖高置信度样本而忽略低置信度样本潜力。本文提出MS-TTA，一种无训练方法，通过单步k近邻（kNN）均值迁移，将特征表示拓展至CLIP原空间之外。该方法对所有测试样本进行特征优化，增强特征紧凑性与类别可分性，实现更稳定的自适应。同时，引入优化嵌入缓存，进一步提升推理性能。在多种分布外及跨数据集基准上评估表明，MS-TTA持续优于当前最优无训练TTA方法，实现稳健自适应且无需额外训练。
原文摘要 · Abstract (English)
Visual-language models (VLMs) like CLIP exhibit strong generalization but struggle with distribution shifts at test time. Existing training-free test-time adaptation (TTA) methods operate strictly within CLIP's original feature space, relying on high-confidence samples while overlooking the potential of low-confidence ones. We propose MS-TTA, a training-free approach that enhances feature representations beyond CLIP's space using a single-step k-nearest neighbors (kNN) Mean-Shift. By refining all test samples, MS-TTA improves feature compactness and class separability, leading to more stable adaptation. Additionally, a cache of refined embeddings further enhances inference by providing Mean Shift enhanced logits. Extensive evaluations on OOD and cross-dataset benchmarks demonstrate that MS-TTA consistently outperforms state-of-the-art training-free TTA methods, achieving robust adaptation without requiring additional training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。