arXiv:2607.09785cs.CVcs.AI2026-07中稿 · publication in IEE…综述

提出视觉持续自监督学习系统综述,解决无标签场景下模型长期适应难题。

Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models

论文配图:Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models
图 1 · 摘自论文原文
  • 按遗忘缓解策略分类现有方法,涵盖蒸馏、重放、正则化等六类
  • 发现自监督目标因任务无关表征更抗灾难性遗忘
  • 适合研究持续学习与大规模预训练的科研人员参考

传统持续学习依赖标注数据,但真实场景如终身机器人需从无标签数据流中持续适应。为此,持续自监督学习(CSSL)迅速发展,却缺乏系统性综述。本文对视觉领域的CSSL进行全面梳理,关联新兴的视觉-语言场景。首先分析现有评估协议,指出不一致问题影响公平比较;其次揭示自监督目标因具备任务无关表征和更平滑的损失曲面,能更好抵御灾难性遗忘;接着基于遗忘缓解策略构建统一分类体系,包括知识蒸馏、经验回放、正则化、架构设计、模型融合及目标级适配;最后指出现有挑战,如可扩展性与快速适应需求。文章主张推进CSSL需超越小规模基准,迈向大规模系统的持续预训练范式。

原文摘要 · Abstract (English)

Traditionally, continual learning has assumed access to labeled data, yet many real-world applications -- such as lifelong robotics -- require models to adapt continuously from unlabeled streams. This has led to the development of continual self-supervised learning (CSSL), a rapidly growing area that lacks a dedicated, systematic review. In this work, we present a comprehensive survey of CSSL for vision, with connections to emerging vision-language settings. First, we analyze existing evaluation protocols and highlight inconsistencies that hinder fair comparison. We then examine why self-supervised objectives exhibit improved robustness to catastrophic forgetting, relating this to task-agnostic representations and smoother loss landscapes. Next, we organize existing methods into a unified taxonomy based on their forgetting-mitigation strategies, including distillation, replay, regularization, architectural approaches, model merging, and objective-level adaptation. Finally, we identify open challenges such as scalability and the need for fast adaptability. We argue that advancing CSSL requires moving beyond small-scale benchmarks towards continual pre-training paradigms for large-scale systems.

持续学习自监督视觉模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。