无监督持续聚类新方法,避免遗忘且不存旧数据
Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation

- 用前后双向知识蒸馏让模型边学新任务边保留旧聚类结构
- 在4个数据集上聚类准确率超越现有方法,遗忘显著减少
- 适合长期在线学习、注重隐私与内存效率的场景
无监督持续学习(UCL)旨在使神经网络在无标签且无法访问历史数据的情况下持续学习序列化任务。其主要挑战是灾难性遗忘——模型在学习新任务时会遗忘旧任务。这一问题在无监督场景下尤为严峻,因缺乏标签引导学习与记忆保持。现有缓解策略如知识蒸馏和回放缓冲区常引发内存与隐私问题。此外,当前方法大多忽视聚类特定目标。为此,本文提出无监督持续聚类(UCC),并设计面向持续聚类的前向-后向知识蒸馏(FBCC)。FBCC采用带聚类投影器的持续教师网络与轻量级任务专属学生网络,通过双阶段前向-后向蒸馏过程,在不存储过去数据的前提下学习新聚类并保留原有聚类结构。该方法为首个面向UCC的系统性方案,在四个基准数据集上的实验表明,其聚类准确率持续优于现有持续学习基线,且显著缓解灾难性遗忘。
原文摘要 · Abstract (English)
Unsupervised Continual Learning (UCL) aims to enable neural networks to learn sequential tasks without labels or access to past data. A major challenge in this setting is Catastrophic Forgetting, where models forget previously learned tasks upon learning new ones. This challenge is amplified in UCL due to the absence of labels to guide learning and memory retention. Existing mitigation strategies, such as knowledge distillation and replay buffers, often raise memory and privacy concerns. Moreover, current UCL methods largely overlook clustering-specific objectives. To fill this gap, we introduce Unsupervised Continual Clustering (UCC) and propose Forward-Backward Knowledge Distillation for Continual Clustering (FBCC). FBCC employs a continual teacher network with a clustering projector and lightweight task-specific students. Through a dual-phase forward-backward distillation process, the teacher learns new clusters while preserving previously discovered cluster structure without storing past data. FBCC represents a pioneering approach to UCC, demonstrating improved clustering performance across sequential tasks. Experiments on four benchmark datasets demonstrate that FBCC consistently outperforms existing continual learning baselines in clustering accuracy while significantly reducing catastrophic forgetting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。