用原型记忆防止深度补全模型遗忘旧知识，支持持续学习。
ProtoDepth: Unsupervised Continual Depth Completion with Prototypes
- 用可调原型集适配新场景，冻结原模型避免遗忘
- 在室内和室外数据上遗忘率分别降低52.2%和53.2%
- 适合长期运行的自动驾驶等需要持续学习的场景
我们提出ProtoDepth，一种基于原型的无监督持续深度补全方法，旨在从RGB图像和稀疏点云中重建稠密深度图。该任务属于多模态三维重建。无监督学习天然适合持续学习，因无需真实标签。然而，在面对非平稳分布的新数据时，模型会灾难性遗忘先前知识。为解决此问题，我们学习一组可适应新域的原型集，用于调整冻结预训练模型的隐空间特征。由于原始权重不变，当测试域已知时，ProtoDepth不会遗忘。针对测试域未知的挑战性场景，我们进一步学习域描述符，使模型能自动选择合适原型集进行推理。在基准数据集序列上评估，相比基线，室内与室外场景的遗忘率分别降低52.2%和53.2%，达到当前最优性能。
原文摘要 · Abstract (English)
We present ProtoDepth, a novel prototype-based approach for continual learning of unsupervised depth completion, the multimodal 3D reconstruction task of predicting dense depth maps from RGB images and sparse point clouds. The unsupervised learning paradigm is well-suited for continual learning, as ground truth is not needed. However, when training on new non-stationary distributions, depth completion models will catastrophically forget previously learned information. We address forgetting by learning prototype sets that adapt the latent features of a frozen pretrained model to new domains. Since the original weights are not modified, ProtoDepth does not forget when test-time domain identity is known. To extend ProtoDepth to the challenging setting where the test-time domain identity is withheld, we propose to learn domain descriptors that enable the model to select the appropriate prototype set for inference. We evaluate ProtoDepth on benchmark dataset sequences, where we reduce forgetting compared to baselines by 52.2% for indoor and 53.2% for outdoor to achieve the state of the art.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。