提出新归一化方法,让深度补全模型更适应未知场景。
Scale Propagation Network for Generalizable Depth Completion
- 用可学习的缩放机制替代传统归一化,保留尺度信息
- 在6个未见数据集上均超越现有方法,精度更高速度更快
- 适合需要跨场景泛化的深度补全应用
深度补全旨在从稀疏测量中推断稠密深度图,对鲁棒三维感知至关重要。尽管基于深度学习的方法已取得显著进展,但这些模型在训练中未见过的场景上泛化能力差,仍是根本性挑战。分析发现,现有网络架构多借鉴图像分析任务的骨干结构,其关键瓶颈在于传统归一化层:它们虽提升训练稳定性并增强视觉不变性,却削弱了对实际尺度的估计能力。为此,本文提出一种新的尺度传播归一化(SP-Norm),通过学习单层感知机对归一化输入进行缩放,实现尺度从输入到输出的传递,同时保持归一化以确保收敛。在此基础上,结合ConvNeXt V2骨干构建新网络架构,优化基本模块组合以实现高性能与高效推理。在六个未见数据集上进行实验,涵盖0.1%/1%/10%随机采样、4/8/16/32/64线激光雷达点以及结构光生成的孔洞,模型在各项指标上均优于当前最优方法,精度更高,速度更快,内存更低。
原文摘要 · Abstract (English)
Depth completion, inferring dense depth maps from sparse measurements, is crucial for robust 3D perception. Although deep learning based methods have made tremendous progress in this problem, these models cannot generalize well across different scenes that are unobserved in training, posing a fundamental limitation that yet to be overcome. A careful analysis of existing deep neural network architectures for depth completion, which are largely borrowing from successful backbones for image analysis tasks, reveals that a key design bottleneck actually resides in the conventional normalization layers. These normalization layers are designed, on one hand, to make training more stable, on the other hand, to build more visual invariance across scene scales. However, in depth completion, the scale is actually what we want to robustly estimate in order to better generalize to unseen scenes. To mitigate, we propose a novel scale propagation normalization (SP-Norm) method to propagate scales from input to output, and simultaneously preserve the normalization operator for easy convergence. More specifically, we rescale the input using learned features of a single-layer perceptron from the normalized input, rather than directly normalizing the input as conventional normalization layers. We then develop a new network architecture based on SP-Norm and the ConvNeXt V2 backbone. We explore the composition of various basic blocks and architectures to achieve superior performance and efficient inference for generalizable depth completion. Extensive experiments are conducted on six unseen datasets with various types of sparse depth maps, i.e., randomly sampled 0.1\%/1\%/10\% valid pixels, 4/8/16/32/64-line LiDAR points, and holes from Structured-Light. Our model consistently achieves the best accuracy with faster speed and lower memory when compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。