提升单目深度估计中位姿与深度的尺度一致性，无需增加推理时间。
SA4Depth: Consistent Pose-Depth Scale Alignment for Self-Supervised Monocular Depth Estimation
- 用深度预测重投影可学习特征，优化位姿估计以对齐尺度。
- 在KITTI、Cityscapes等数据集上深度预测精度显著提升。
- 适合需要高精度深度估计的自动驾驶与机器人应用。
单目序列的自监督深度估计依赖于深度网络与位姿网络的联合学习。尽管深度网络研究丰富，但位姿网络改进有限。本文指出：即使深度仅能估计到比例尺度，位姿与深度网络所估计的场景尺度对齐仍至关重要。为此提出SA4Depth方法，在不增加推理时间的前提下，利用训练时的深度预测结果，将可学习视觉特征在连续帧间重投影，并通过减少特征对齐残差来精修位姿估计。该方法使深度与位姿网络估算的场景尺度一致，提升了不同序列间的预测尺度一致性。其可微分优化模块可无缝集成至现有自监督流程，显著改善深度估计性能。实验在室外（KITTI、Cityscapes）和室内（NYUv2）数据集上验证了有效性，且在KITTI Odometry上证实了位姿精修效果。代码已开源。
原文摘要 · Abstract (English)
Self-supervised depth estimation from monocular sequences relies on the joint learning of a depth and a pose network. Despite abundant research done to improve the depth network, efforts on the pose remain limited. In this context, even when depth is estimated up to scale, we highlight the importance of the alignment between the scene scales estimated by the pose and depth nets. Then, we introduce SA4Depth, an approach to improve this alignment and boost the depth predictions while keeping the inference time unchanged. Our proposed method uses the depth estimated during training to reproject learnable visual features across consecutive frames and refine the pose estimates by reducing feature alignment residuals. With our method, the estimated scene scales by the separate depth and pose networks are aligned, and the prediction scale consistency is improved across different sequences. Our differentiable refinement integrates seamlessly into existing self-supervised pipelines and substantially improves their depth estimates. We demonstrate this with extensive experiments both outdoors and indoors on KITTI, Cityscapes, and NYUv2. Additionally, results on KITTI Odometry confirm the effectiveness of our pose refinement. Our code is available at https://github.com/Runningchauncey/SA4Depth .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。