通过隐空间重中心化实现无需优化的高效测试时自适应
NEO: No-Optimization Test-Time Adaptation through Latent Re-Centering
- 将目标数据嵌入重置到原点，提升源域与分布偏移样本对齐度
- 仅用64个样本即使ViT-Base在ImageNet-C上准确率从55.6%提至59.2%
- 无超参、低计算开销，适合边缘设备部署
测试时自适应（TTA）方法通常计算开销大、需大量数据或对超参数敏感。基于隐空间几何理论，我们发现将目标数据嵌入重置到原点可显著提升源域与分布偏移样本的对齐度。据此提出NEO——一种无超参的全测试时自适应方法，额外计算开销可忽略。在仅使用64个样本的情况下，NEO使ViT-Base在ImageNet-C上的分类准确率从55.6%提升至59.2%。使用512个样本时，NEO在ImageNet-C、ImageNet-R、ImageNet-S上超越全部7种对比的TTA方法，在CIFAR-10-C上优于6/7，且计算量最低。NEO在模型校准指标上表现良好，并能从1类样本迁移以提升999类其他类别的准确率。在Raspberry Pi和Jetson Orin Nano设备上，相比基线，推理时间减少63%，内存占用降低9%。基于3种ViT架构和4个数据集的实验表明，NEO可高效、有效用于测试时自适应。
原文摘要 · Abstract (English)
Test-Time Adaptation (TTA) methods are often computationally expensive, require a large amount of data for effective adaptation, or are brittle to hyperparameters. Based on a theoretical foundation of the geometry of the latent space, we are able to significantly improve the alignment between source and distribution-shifted samples by re-centering target data embeddings at the origin. This insight motivates NEO -- a hyperparameter-free fully TTA method, that adds no significant compute compared to vanilla inference. NEO is able to improve the classification accuracy of ViT-Base on ImageNet-C from 55.6% to 59.2% after adapting on just one batch of 64 samples. When adapting on 512 samples NEO beats all 7 TTA methods we compare against on ImageNet-C, ImageNet-R and ImageNet-S and beats 6/7 on CIFAR-10-C, while using the least amount of compute. NEO performs well on model calibration metrics and additionally is able to adapt from 1 class to improve accuracy on 999 other classes in ImageNet-C. On Raspberry Pi and Jetson Orin Nano devices, NEO reduces inference time by 63% and memory usage by 9% compared to baselines. Our results based on 3 ViT architectures and 4 datasets show that NEO can be used efficiently and effectively for TTA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。