解决3D实例分割中类别渐增与数据不平衡问题，提升稀有类表现。
CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation
- 用伪标签生成+动态重加权缓解记忆受限下的类别偏差
- 在ScanNet200上实例分割mAP提升最高达16.76%
- 适合持续学习场景下需兼顾稀有类性能的研究者
尽管3D实例分割（3DIS）进展显著，但现有方法通常假设所有类别预先已知且分布均匀，这在真实动态环境中不成立。新类别逐渐出现且存在自然不平衡，现有方法多忽略此问题，导致稀有类别性能下降。为此，我们提出统一框架CLIMB-3D，支持类别增量与不平衡感知的3DIS。基于经典示例回放（ER）策略，发现其在内存约束下效果不足。为此引入伪标签生成器（PLG），利用冻结模型对旧类别生成扩展监督。但PLG易偏向常见类别，因此提出类别平衡重加权（CBR）机制，从伪标签估计类别频率并动态调整训练权重，无需访问历史数据。在挑战性的ScanNet200上设计三种增量场景评估，并在ScanNetV2上验证语义分割性能。结果表明，本方法在实例分割上相比先前工作最高提升16.76% mAP，语义分割提升约30% mIoU，展现出对高频与低频类别的良好泛化能力。代码开源：https://github.com/vgthengane/CLIMB3D
原文摘要 · Abstract (English)
While 3D instance segmentation (3DIS) has advanced significantly, most existing methods assume that all object classes are known in advance and uniformly distributed. However, this assumption is unrealistic in dynamic, real-world environments where new classes emerge gradually and exhibit natural imbalance. Although some approaches address the emergence of new classes, they often overlook class imbalance, which leads to suboptimal performance, particularly on rare categories. To tackle this, we propose \ourmethodbf, a unified framework for \textbf{CL}ass-incremental \textbf{Imb}alance-aware \textbf{3D}IS. Building upon established exemplar replay (ER) strategies, we show that ER alone is insufficient to achieve robust performance under memory constraints. To mitigate this, we introduce a novel pseudo-label generator (PLG) that extends supervision to previously learned categories by leveraging predictions from a frozen model trained on prior tasks. Despite its promise, PLG tends to be biased towards frequent classes. Therefore, we propose a class-balanced re-weighting (CBR) scheme that estimates object frequencies from pseudo-labels and dynamically adjusts training bias, without requiring access to past data. We design and evaluate three incremental scenarios for 3DIS on the challenging ScanNet200 dataset and additionally validate our method for semantic segmentation on ScanNetV2. Our approach achieves state-of-the-art results, surpassing prior work by up to 16.76\% mAP for instance segmentation and approximately 30\% mIoU for semantic segmentation, demonstrating strong generalisation across both frequent and rare classes. Code is available at: https://github.com/vgthengane/CLIMB3D
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。