提出ASR策略,让模型长期适应变化环境时保持学习能力。
Maintain Plasticity in Long-timescale Continual Test-time Adaptation
- 根据标签翻转变化动态调整权重重置周期,维持模型适应力
- 实验证明传统方法长期适应中学习能力持续下降,而本方法显著缓解
- 适合需要长期稳定适应新环境的工业级部署场景
持续测试时领域自适应(CTTA)旨在使预训练模型在非平稳目标环境中随时间持续优化性能。尽管已有方法致力于优化适应过程,但一个关键问题仍未解决:模型能否在长时间尺度上持续适应不断变化的环境,同时保持学习能力(即塑性)?本文首次系统研究塑性这一长期适应中的核心但常被忽视的特性。实验发现,大多数现有方法在长期连续适应阶段均呈现塑性持续下降趋势,且其损失与标签翻转频率密切相关。基于此关联,本文提出一种简单有效的策略——自适应收缩-恢复(ASR),通过根据标签翻转变化动态确定权重重置间隔实现塑性保持。该方法在多个主流CTTA基准上验证,表现优异。
原文摘要 · Abstract (English)
Continual test-time domain adaptation (CTTA) aims to adjust pre-trained source models to perform well over time across non-stationary target environments. While previous methods have made considerable efforts to optimize the adaptation process, a crucial question remains: can the model adapt to continually-changing environments with preserved plasticity over a long time? The plasticity refers to the model's capability to adjust predictions in response to non-stationary environments continually. In this work, we explore plasticity, this essential but often overlooked aspect of continual adaptation to facilitate more sustained adaptation in the long run. First, we observe that most CTTA methods experience a steady and consistent decline in plasticity during the long-timescale continual adaptation phase. Moreover, we find that the loss of plasticity is strongly associated with the change in label flip. Based on this correlation, we propose a simple yet effective policy, Adaptive Shrink-Restore (ASR), towards preserving the model's plasticity. In particular, ASR does the weight re-initialization by the adaptive intervals. The adaptive interval is determined based on the change in label flipping. Our method is validated on extensive CTTA benchmarks, achieving excellent performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。