解耦检测与分割,提升持续语义分割的遗忘抑制能力
Decoupling Continual Semantic Segmentation
- 分两阶段设计:先用文本图像编码器生成类特定提示,再用SAM做精准分割
- 在COCO-20k和PASCAL-Context上达到新最优,旧类别记忆保持率超90%
- 适合关注长期学习、减少灾难性遗忘的视觉模型研究者
持续语义分割(CSS)需在不遗忘旧知识的前提下学习新类别,应对密集预测任务中的灾难性遗忘问题。现有方法多采用单阶段编码器-解码器结构,使分割掩码与类别标签紧密耦合,导致新旧类别学习相互干扰,保留与适应的平衡不佳。本文提出DecoupleCSS,一种新型两阶段框架。通过解耦类感知检测与类无关分割,实现更有效的持续学习,兼顾旧知识保留与新类学习。第一阶段利用预训练文本与图像编码器,结合LoRA进行微调,生成带位置信息的类特定提示;第二阶段采用分割任意模型(SAM)生成高精度分割掩码,确保分割知识在新旧类别间共享。该方法显著改善了保留与适应的平衡,在多个挑战性任务中取得当前最优性能。代码已公开于https://github.com/euyis1019/Decoupling-Continual-Semantic-Segmentation。
原文摘要 · Abstract (English)
Continual Semantic Segmentation (CSS) requires learning new classes without forgetting previously acquired knowledge, addressing the fundamental challenge of catastrophic forgetting in dense prediction tasks. However, existing CSS methods typically employ single-stage encoder-decoder architectures where segmentation masks and class labels are tightly coupled, leading to interference between old and new class learning and suboptimal retention-plasticity balance. We introduce DecoupleCSS, a novel two-stage framework for CSS. By decoupling class-aware detection from class-agnostic segmentation, DecoupleCSS enables more effective continual learning, preserving past knowledge while learning new classes. The first stage leverages pre-trained text and image encoders, adapted using LoRA, to encode class-specific information and generate location-aware prompts. In the second stage, the Segment Anything Model (SAM) is employed to produce precise segmentation masks, ensuring that segmentation knowledge is shared across both new and previous classes. This approach improves the balance between retention and adaptability in CSS, achieving state-of-the-art performance across a variety of challenging tasks. Our code is publicly available at: https://github.com/euyis1019/Decoupling-Continual-Semantic-Segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。