提出SOLAR方法，解决在线持续自监督学习中的隐空间衰退问题。
Preventing Latent Rehearsal Decay in Online Continual SSL with SOLAR
- 用新指标诊断隐空间退化，动态调节记忆缓冲区管理。
- 在视觉基准上实现最快收敛与最高最终性能。
- 适合需要快速适应新数据且保持长期稳定性的场景。
本文研究在线持续自监督学习（OCSSL），即模型从连续的非平稳无标签数据流中学习。现有方法多依赖重放机制，但稳定性与可塑性权衡导致性能下降。我们提出隐空间重放衰减假说，解释性能崩溃源于过度稳定的重放引发的隐空间退化。引入重叠度（Overlap）和偏差（Deviation）两个指标诊断退化，并与准确率下降相关联。基于此，提出SOLAR方法，利用偏差的在线代理指导缓冲区管理，结合显式的重叠损失，实现可塑性自适应。实验表明，SOLAR在多个视觉OCSSL基准上达到当前最优表现，兼具快速收敛与高最终精度。
原文摘要 · Abstract (English)
This paper explores Online Continual Self-Supervised Learning (OCSSL), a scenario in which models learn from continuous streams of unlabeled, non-stationary data, where methods typically employ replay and fast convergence is a central desideratum. We find that OCSSL requires particular attention to the stability-plasticity trade-off: stable methods (e.g. replay with Reservoir sampling) are able to converge faster compared to plastic ones (e.g. FIFO buffer), but incur in performance drops under certain conditions. We explain this collapse phenomenon with the Latent Rehearsal Decay hypothesis, which attributes it to latent space degradation under excessive stability of replay. We introduce two metrics (Overlap and Deviation) that diagnose latent degradation and correlate with accuracy declines. Building on these insights, we propose SOLAR, which leverages efficient online proxies of Deviation to guide buffer management and incorporates an explicit Overlap loss, allowing SOLAR to adaptively managing plasticity. Experiments demonstrate that SOLAR achieves state-of-the-art performance on OCSSL vision benchmarks, with both high convergence speed and final performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。