通过跨任务对齐提升测试时训练的鲁棒性
CTA: Cross-Task Alignment for Better Test Time Training
- 用对比学习思想对齐监督与自监督编码器
- 在多个基准数据集上显著提升泛化性能
- 无需特殊架构，适合部署于现有模型
深度学习模型在计算机视觉任务中表现优异，但在分布偏移（如领域或数据集变化）下性能常显著下降。测试时训练（TTT）通过在训练中引入辅助无监督任务，并在测试时利用该任务更新模型，有效增强模型鲁棒性。本文提出一种新方法CTA（Cross-Task Alignment），不依赖专用模型结构，借鉴多模态对比学习的成功经验，将监督编码器与自监督编码器的特征表示对齐。该过程强化了两模型间表示的一致性，缓解梯度干扰，保留自监督学习的内在鲁棒性，使测试时更新更具语义意义。实验表明，CTA在多个基准数据集上显著优于当前最优方法，大幅提升了模型的鲁棒性与泛化能力。
原文摘要 · Abstract (English)
Deep learning models have demonstrated exceptional performance across a wide range of computer vision tasks. However, their performance often degrades significantly when faced with distribution shifts, such as domain or dataset changes. Test-Time Training (TTT) has emerged as an effective method to enhance model robustness by incorporating an auxiliary unsupervised task during training and leveraging it for model updates at test time. In this work, we introduce CTA (Cross-Task Alignment), a novel approach for improving TTT. Unlike existing TTT methods, CTA does not require a specialized model architecture and instead takes inspiration from the success of multi-modal contrastive learning to align a supervised encoder with a self-supervised one. This process enforces alignment between the learned representations of both models, thereby mitigating the risk of gradient interference, preserving the intrinsic robustness of self-supervised learning and enabling more semantically meaningful updates at test-time. Experimental results demonstrate substantial improvements in robustness and generalization over the state-of-the-art on several benchmark datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。