通过测试时相关性对齐,无需源数据即可提升模型泛化性能。
Test-time Correlation Alignment
- 利用高置信度样本与测试样本的相关性对齐,实现无模型更新的适应。
- 相比最优基线,仅需4%显存和0.6%计算时间,准确率显著提升。
- 适用于隐私敏感场景,可无缝集成到现有测试时自适应方法中。
深度神经网络在分布偏移下性能常下降。尽管领域自适应提供解决方案,但隐私限制常使源数据不可访问,因此仅使用无标签测试数据的测试时自适应(TTA)日益重要。然而现有方法仍面临三大挑战:(1) 主要关注实例级对齐,忽略因缺少源数据相关性而无法实现的相关性对齐(CORAL);(2) 模型更新依赖复杂反向传播,带来额外计算开销;(3) 存在领域遗忘问题。为此,我们从理论上分析了测试时相关性对齐(TCA)的可行性,证明高置信度实例与测试实例间的相关性对齐可在理论上提升测试性能。基于此,提出两种简单高效算法:LinearTCA 和 LinearTCA+。LinearTCA 通过简单线性变换实现实例与相关性对齐,无需额外模型更新;LinearTCA+ 可作为即插即用模块,轻松增强现有 TTA 方法。大量实验验证了理论有效性,结果表明 TCA 方法在多种任务、基准和主干网络上均显著优于基线。特别地,LinearTCA 在保持仅 4% GPU 内存和 0.6% 计算时间的前提下,超越最佳基线,并在 CLIP 上提升超 1.86%。
原文摘要 · Abstract (English)
Deep neural networks often degrade under distribution shifts. Although domain adaptation offers a solution, privacy constraints often prevent access to source data, making Test-Time Adaptation (TTA, which adapts using only unlabeled test data) increasingly attractive. However, current TTA methods still face practical challenges: (1) a primary focus on instance-wise alignment, overlooking CORrelation ALignment (CORAL) due to missing source correlations; (2) complex backpropagation operations for model updating, resulting in overhead computation and (3) domain forgetting. To address these challenges, we provide a theoretical analysis to investigate the feasibility of Test-time Correlation Alignment (TCA), demonstrating that correlation alignment between high-certainty instances and test instances can enhance test performances with a theoretical guarantee. Based on this, we propose two simple yet effective algorithms: LinearTCA and LinearTCA+. LinearTCA applies a simple linear transformation to achieve both instance and correlation alignment without additional model updates, while LinearTCA+ serves as a plug-and-play module that can easily boost existing TTA methods. Extensive experiments validate our theoretical insights and show that TCA methods significantly outperforms baselines across various tasks, benchmarks and backbones. Notably, LinearTCA achieves higher accuracy with only 4% GPU memory and 0.6% computation time compared to the best TTA baseline. It also outperforms existing methods on CLIP over 1.86%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。