arXiv:2507.07831cs.CV2025-07CVPR被引 19

提出 SimCIS 框架,解决持续图像分割中遗忘与数据顺序依赖问题。

Rethinking Query-based Transformer for Continual Image Segmentation

  • 直接用图像特征匹配查询,实现精准对齐并保留对象感知能力。
  • 在多个任务和数据顺序下均超越现有方法,平均性能提升超 5%。
  • 适合研究持续学习、图像分割的开发者,代码已开源。

类增量/持续图像分割(CIS)旨在分阶段训练图像分割模型,每阶段可用类别集不同。现有基于查询的变换器方法常将掩码生成与持续学习过程解耦,以利用其内置的对象感知能力,缓解掩码提议的灾难性遗忘。本文指出此类解耦框架存在两大问题:可塑性丧失与对输入数据顺序的高度依赖。通过深入分析内置对象感知机制,发现高度聚合的图像特征为查询提供简单特征对齐的捷径以生成掩码。基于此,提出 SimCIS——一种简单但强大的 CIS 基线。其核心思想是直接选择图像特征分配给查询,确保“完美对齐”以保持对象感知,同时允许查询主动选择新类别以促进可塑性。为进一步缓解类别遗忘,引入跨阶段选择一致性及创新的“视觉查询”回放机制。实验表明,SimCIS 在多种分割任务、设置、数据划分及输入顺序下均稳定优于当前最优方法。所有模型与代码将公开于 https://github.com/SooLab/SimCIS。

原文摘要 · Abstract (English)

Class-incremental/Continual image segmentation (CIS) aims to train an image segmenter in stages, where the set of available categories differs at each stage. To leverage the built-in objectness of query-based transformers, which mitigates catastrophic forgetting of mask proposals, current methods often decouple mask generation from the continual learning process. This study, however, identifies two key issues with decoupled frameworks: loss of plasticity and heavy reliance on input data order. To address these, we conduct an in-depth investigation of the built-in objectness and find that highly aggregated image features provide a shortcut for queries to generate masks through simple feature alignment. Based on this, we propose SimCIS, a simple yet powerful baseline for CIS. Its core idea is to directly select image features for query assignment, ensuring "perfect alignment" to preserve objectness, while simultaneously allowing queries to select new classes to promote plasticity. To further combat catastrophic forgetting of categories, we introduce cross-stage consistency in selection and an innovative "visual query"-based replay mechanism. Experiments demonstrate that SimCIS consistently outperforms state-of-the-art methods across various segmentation tasks, settings, splits, and input data orders. All models and codes will be made publicly available at https://github.com/SooLab/SimCIS.

持续学习图像分割变压器查询机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。