动态检测领域偏移，智能切换适配策略提升模型鲁棒性
Hybrid-TTA: Continual Test-time Adaptation via Dynamic Domain Shift Detection
- 根据输入序列时序相关性动态判断领域偏移，选择全微调或高效微调
- 在Cityscapes转ACDC任务上实现mIoU提升1.6个百分点
- 适合需要持续适应真实场景的视觉模型部署
持续测试时自适应（CTTA）已成为弥合受控训练环境与真实世界场景间领域差距的关键方法，提升模型适应性与鲁棒性。现有CTTA方法通常分为全微调（FT）和高效微调（ET），难以有效应对领域偏移。为此，我们提出Hybrid-TTA，一种通过动态领域偏移检测（DDSD）策略实现实例级调优方法选择的综合方案。该策略利用输入序列的时序相关性识别领域偏移，并动态在FT与ET间切换，以有效适应不同变化。此外，集成基于掩码图像建模的适配框架（MIMA），在最小计算开销下实现领域无关的鲁棒性。Hybrid-TTA在Cityscapes-to-ACDC基准数据集上取得1.6个百分点的mIoU提升，超越现有最先进方法，为真实世界持续适应挑战提供稳健解决方案。
原文摘要 · Abstract (English)
Continual Test Time Adaptation (CTTA) has emerged as a critical approach for bridging the domain gap between the controlled training environments and the real-world scenarios, enhancing model adaptability and robustness. Existing CTTA methods, typically categorized into Full-Tuning (FT) and Efficient-Tuning (ET), struggle with effectively addressing domain shifts. To overcome these challenges, we propose Hybrid-TTA, a holistic approach that dynamically selects instance-wise tuning method for optimal adaptation. Our approach introduces the Dynamic Domain Shift Detection (DDSD) strategy, which identifies domain shifts by leveraging temporal correlations in input sequences and dynamically switches between FT and ET to adapt to varying domain shifts effectively. Additionally, the Masked Image Modeling based Adaptation (MIMA) framework is integrated to ensure domain-agnostic robustness with minimal computational overhead. Our Hybrid-TTA achieves a notable 1.6%p improvement in mIoU on the Cityscapes-to-ACDC benchmark dataset, surpassing previous state-of-the-art methods and offering a robust solution for real-world continual adaptation challenges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。