解决多传感器图像分割中新增模态时的遗忘问题
Modality-Incremental Learning with Disjoint Relevance Mapping Networks for Image-based Semantic Segmentation
- 用不共享连接的相关性映射网络增量学习新传感器数据
- 在严格持续学习框架下有效缓解灾难性遗忘
- 适合自动驾驶中多传感器融合的场景
在自动驾驶中，环境感知得益于相机、深度传感器或红外传感器等多元传感器的深度学习应用。传感器组合的多样性提升了安全性，并增强了对恶劣天气和光照条件的鲁棒性。然而，不同传感器采集的数据差异带来了挑战。在持续学习（CL）背景下，增量学习面临显著的领域偏移问题，例如不同传感器模态，这加剧了灾难性遗忘。为此，我们提出模态增量学习的概念，通过对比现有增量学习范式，验证其必要性。我们采用改进的无关性映射网络（RMN），在保持先前模态性能的同时，增量学习新模态，且相关性映射不共享。实验表明，该方法在严格持续学习框架下，通过避免共享连接有效缓解遗忘问题。
原文摘要 · Abstract (English)
In autonomous driving, environment perception has significantly advanced with the utilization of deep learning techniques for diverse sensors such as cameras, depth sensors, or infrared sensors. The diversity in the sensor stack increases the safety and contributes to robustness against adverse weather and lighting conditions. However, the variance in data acquired from different sensors poses challenges. In the context of continual learning (CL), incremental learning is especially challenging for considerably large domain shifts, e.g. different sensor modalities. This amplifies the problem of catastrophic forgetting. To address this issue, we formulate the concept of modality-incremental learning and examine its necessity, by contrasting it with existing incremental learning paradigms. We propose the use of a modified Relevance Mapping Network (RMN) to incrementally learn new modalities while preserving performance on previously learned modalities, in which relevance maps are disjoint. Experimental results demonstrate that the prevention of shared connections in this approach helps alleviate the problem of forgetting within the constraints of a strict continual learning framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。