用少量样本提升模型在未知数据上的适应能力，减少盲目调整。
Enhancing Test Time Adaptation with Few-shot Guidance
- 引入少量支持样本指导测试时自适应，避免无目标调整。
- 两阶段框架：先微调模型，再用原型记忆库生成高质量伪标签。
- 在三个跨域分类任务中表现更优，适合实际部署场景。
深度神经网络在训练数据（源域）与测试数据（目标域）存在分布偏移时性能显著下降。为解决此问题，测试时自适应（TTA）方法被提出，用于将预训练的源模型适配到分布外的实时目标数据。尽管这些方法有一定缓解作用，但缺乏可靠的分布偏移修正机制，导致在真实应用中表现不稳定。为此，我们提出少样本测试时自适应（FS-TTA），一种新型实用设定，在传统TTA基础上引入少量支持样本。遵循‘少量输入，大幅增益’原则，FS-TTA减少了对未知目标域的盲目探索。进一步提出两阶段框架：(i) 利用少样本支持集微调预训练模型，并结合特征多样性增强模块防止过拟合；(ii) 基于原型记忆库引导的测试时自适应，生成高质量伪标签以指导模型更新。在三个跨域分类基准上进行大量实验，验证了所提方法在性能和可靠性上的优势。
原文摘要 · Abstract (English)
Deep neural networks often encounter significant performance drops while facing with domain shifts between training (source) and test (target) data. To address this issue, Test Time Adaptation (TTA) methods have been proposed to adapt pre-trained source model to handle out-of-distribution streaming target data. Although these methods offer some relief, they lack a reliable mechanism for domain shift correction, which can often be erratic in real-world applications. In response, we develop Few-Shot Test Time Adaptation (FS-TTA), a novel and practical setting that utilizes a few-shot support set on top of TTA. Adhering to the principle of few inputs, big gains, FS-TTA reduces blind exploration in unseen target domains. Furthermore, we propose a two-stage framework to tackle FS-TTA, including (i) fine-tuning the pre-trained source model with few-shot support set, along with using feature diversity augmentation module to avoid overfitting, (ii) implementing test time adaptation based on prototype memory bank guidance to produce high quality pseudo-label for model adaptation. Through extensive experiments on three cross-domain classification benchmarks, we demonstrate the superior performance and reliability of our FS-TTA and framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。