无需训练即可持续学习,避免遗忘并提升旧任务表现
SAILS: Segment Anything with Incrementally Learned Semantics for Task-Invariant and Training-Free Continual Learning
- 用SAM提取区域,通过固定特征空间的原型关联语义
- 零训练下性能超越多数需重训方法,长序列更优
- 适合资源受限场景,特别适合持续新增类别的应用
持续学习受制于重复训练、高计算成本和遗忘问题,严重限制其在真实场景的应用。本文提出SAILS——一种无需训练的类增量语义分割框架,将任务分解为两阶段:利用Segment Anything Model(SAM)进行零样本区域提取,再通过固定特征空间中的原型实现语义关联。SAILS引入选择性类内聚类,每个类别生成多个原型以更好建模类内差异。实验表明,尽管不进行增量训练,SAILS在标准CISS数据集上通常优于现有训练型方法,尤其在长且挑战性强的任务序列中表现更佳。由于不更新参数,完全消除遗忘,保持稳定且任务无关的性能。此外,新类引入还能带来正向回传效应,提升旧类表现。
原文摘要 · Abstract (English)
Continual learning remains constrained by the need for repeated retraining, high computational costs, and the persistent challenge of forgetting. These factors significantly limit the applicability of continuous learning in real-world settings, as iterative model updates require significant computational resources and inherently exacerbate forgetting. We present SAILS -- Segment Anything with Incrementally Learned Semantics, a training-free framework for Class-Incremental Semantic Segmentation (CISS) that sidesteps these challenges entirely. SAILS leverages foundational models to decouple CISS into two stages: Zero-shot region extraction using Segment Anything Model (SAM), followed by semantic association through prototypes in a fixed feature space. SAILS incorporates selective intra-class clustering, resulting in multiple prototypes per class to better model intra-class variability. Our results demonstrate that, despite requiring no incremental training, SAILS typically surpasses the performance of existing training-based approaches on standard CISS datasets, particularly in long and challenging task sequences where forgetting tends to be most severe. By avoiding parameter updates, SAILS completely eliminates forgetting and maintains consistent, task-invariant performance. Furthermore, SAILS exhibits positive backward transfer, where the introduction of new classes can enhance performance on previous classes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。