不依赖数据的实时语义分割持续学习方法,解决新旧类别遗忘问题。
PILOT: A Data-Free Continual Learning Approach for Real-Time Semantic Segmentation via Boundary Guidance
- 通过并行边界分支捕捉新类高频边缘特征,冻结原模型参数。
- 仅用新类数据训练,保持基类mIoU在90%以上,优于主流方法。
- 适合需实时更新语义模型的自动驾驶、机器人场景。
实时语义分割模型在精度与推理速度间取得良好平衡。但在动态现实环境中部署时,常需在不重新训练全量数据的情况下增量学习新类别,这即为持续学习。传统深度学习微调方法因灾难性遗忘而失效,即模型学会新知识却遗忘旧类别。针对此关键问题,本文提出专为PIDNet(当前主流实时语义分割模型)设计的新型持续学习框架PILOT(Parallel Incremental Learning Over Time)。该方法引入轻量级并行导数分支(D-branch),在冻结原分割网络参数的前提下,实时捕获新类别高频率边界信息。仅使用新类别相关数据,显著降低训练开销。实验表明,该方法在有效分割新类的同时,保持基类平均交并比(mIoU)高于90%,显著优于该领域所有主流持续学习方法。整体上,PILOT能有效缓解灾难性遗忘,且对推理延迟影响极小,维持实时性能。
原文摘要 · Abstract (English)
Real-time semantic segmentation models offer an excellent balance between accuracy and inference speed. However, deploying these models in dynamic real world environments often requires the ability to learn novel classes incrementally without retraining on the entire dataset. This capability is known as continual learning. In this regard, the standard fine-tuning methods in deep learning often fail due to catastrophic forgetting, where the model learns new information but forgets previously trained and learned classes. Contributing to this crucial domain, the current paper proposes a novel continual learning framework tailored for PIDNet, which is a widely cited state-of-the-art real-time semantic segmentation model. Our method, PILOT(Parallel Incremental Learning Over Time), introduces a real-time and lightweight strategy by implementing a parallel Derivative-branch (D-branch) designed to capture the high frequency boundary information of novel classes while freezing the trained parameters of the original segmentation network. This novel setup allows the model to adapt to new semantic categories while preserving the knowledge of previously learned classes. By using only data associated with the new class, our model significantly reduces training overhead. Experimental results demonstrate that our approach successfully segments new classes while maintaining high mean Intersection over Union (mIoU) on the original base classes, thereby comfortably outperforming all major continual learning approaches in this domain. Overall, PILOT is shown to effectively mitigate catastrophic forgetting with minimal impact on inference latency, thus maintaining real-time performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。