轻量级网络实现稀疏激光雷达实时深度补全,适合嵌入式设备部署。
EfficientPENet: Real-Time Depth Completion from Sparse LiDAR via Lightweight Multi-Modal Fusion

- 用ConvNeXt替代ResNet,融合RGB与稀疏深度图特征
- 在KITTI上达到631.94mm RMSE,推理速度达48.76 FPS
- 参数量减少3.7倍,适合边缘计算设备
从稀疏激光雷达测量值和对应RGB图像中完成深度补全,是机器人系统实现精确三维感知的前提。现有方法虽在标准基准上精度高,但依赖重型主干网络,难以实现在嵌入式硬件上的实时部署。本文提出EfficientPENet,一种双分支深度补全网络:以现代版ConvNeXt作为主干,引入对稀疏性不变的卷积操作处理深度流,并通过卷积空间传播网络(CSPN)优化预测结果。RGB分支采用ImageNet预训练的ConvNeXt块,结合层归一化、7×7深度可分离卷积及随机深度正则化。两分支特征经晚期融合后,通过多尺度深度监督策略解码。此外,提出位置感知的测试时增强方案,在水平翻转时修正坐标张量,提升推理一致性。在KITTI深度补全基准上,EfficientPENet实现631.94 mm RMSE,仅需36.24M参数,延迟20.51 ms,运行速度达48.76 FPS。相比BP-Net,参数量减少3.7倍,速度提升23倍,同时保持竞争力精度,成为资源受限边缘平台(如NVIDIA Jetson)上实用的实时深度补全方案。
原文摘要 · Abstract (English)
Depth completion from sparse LiDAR measurements and corresponding RGB images is a prerequisite for accurate 3D perception in robotic systems. Existing methods achieve high accuracy on standard benchmarks but rely on heavy backbone architectures that preclude real-time deployment on embedded hardware. We present EfficientPENet, a two-branch depth completion network that replaces the conventional ResNet encoder with a modernized ConvNeXt backbone, introduces sparsity-invariant convolutions for the depth stream, and refines predictions through a Convolutional Spatial Propagation Network (CSPN). The RGB branch leverages ImageNet-pretrained ConvNeXt blocks with Layer Normalization, 7x7 depthwise convolutions, and stochastic depth regularization. Features from both branches are merged via late fusion and decoded through a multi-scale deep supervision strategy. We further introduce a position-aware test-time augmentation scheme that corrects coordinate tensors during horizontal flipping, yielding consistent error reduction at inference. On the KITTI depth completion benchmark, EfficientPENet achieves an RMSE of 631.94 mm with 36.24M parameters and a latency of 20.51 ms, operating at 48.76 FPS. This represents a 3.7 times reduction in parameters and a 23 times speedup relative to BP-Net, while maintaining competitive accuracy. These results establish EfficientPENet as a practical solution for real-time depth completion on resource-constrained edge platforms such as the NVIDIA Jetson.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。