让模型主动遗忘，再通过测试时训练找回当前域，适合持续视频学习。
Remembering by Reconstructing: Domain Incremental Learning With Test-Time Training on Video Streams
- 用自监督重建机制与LoRA适配器实现域增量学习
- 在动作识别和语义分割上达到优于基线的准确率
- 适合视频流等连续相关数据，支持在线更新
本文提出一种新型域增量学习方法，用于适应随时间变化的非平稳数据。不同于传统方法试图避免灾难性遗忘，本方法允许遗忘并加以利用。模型包含主任务头与自监督掩码自动编码器（MAE）头，增量训练中为各域学习专用的LoRA适配器，使各头对其他域产生自然遗忘。推理时，通过对自监督MAE头进行在线测试时训练，识别出最匹配当前输入的LoRA，从而重新“记忆”该域。该方案特别适用于视频等连续样本高度相关的现实数据流，其中域变化渐进。我们在域增量动作识别与语义分割任务上验证了方法的有效性。
原文摘要 · Abstract (English)
In this work we introduce a novel approach to domain incremental learning, adapting models over time to evolving, non-stationary data. In contrast to other works, we do not attempt to avoid catastrophic forgetting, but rather allow it and exploit it. Our model combines a main task head with a self-supervised masked autoencoder (MAE) head. We then learn domain-specific LoRA adapters during incremental training. Each adapter specializes to its domain, naturally inducing forgetting on other domains in both heads. At inference, we perform online test-time training on the self-supervised MAE head to identify which LoRAs best matches the current input, so the model can `remember' the domain again. Our scheme is especially well-suited to real-world streaming data, such as video, where consecutive samples are highly correlated and domain shifts are gradual. We demonstrate our method on domain-incremental action recognition and semantic segmentation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。