无需额外数据,仅用车载摄像头视频就可生成精确距离的3D场景图。
MonoPP: Metric-Scaled Self-Supervised Monocular Depth Estimation by Planar-Parallax Geometry in Automotive Applications
- 利用平面视差几何与摄像头位置信息,从单目视频中推断真实尺度深度
- 在KITTI上达到当前最佳效果,首次实现城市街景数据集上的自监督真实尺度深度预测
- 适合自动驾驶系统开发,尤其关注低成本视觉感知的团队
自监督单目深度估计(MDE)近年来因能直接从视频中获取深度预测而受到关注。但这类方法通常输出尺度不变的结果,除非引入额外训练信号。为此,本文提出一种新颖的自监督真实尺度单目深度估计模型,仅需单目视频和摄像头安装位置,二者在现代车辆中均易获取。该方法利用平面视差几何重建场景结构,整个流程包含三个主要网络:多帧网络、单帧网络和位姿网络。多帧网络通过序列帧与摄像头安装位置,利用平面视差几何估计静态场景结构,并作为教师网络,将尺度信息、可行驶区域掩码、真实尺度深度及动态物体掩码等知识蒸馏至单帧网络;同时辅助位姿网络预测两帧间的真实尺度相对位姿。本方法在驾驶基准数据集KITTI上达到当前最优表现,且为首个实现挑战性城市街景数据集Cityscapes上自监督真实尺度深度预测的方法,验证了其有效性与通用性。
原文摘要 · Abstract (English)
Self-supervised monocular depth estimation (MDE) has gained popularity for obtaining depth predictions directly from videos. However, these methods often produce scale invariant results, unless additional training signals are provided. Addressing this challenge, we introduce a novel self-supervised metric-scaled MDE model that requires only monocular video data and the camera's mounting position, both of which are readily available in modern vehicles. Our approach leverages planar-parallax geometry to reconstruct scene structure. The full pipeline consists of three main networks, a multi-frame network, a singleframe network, and a pose network. The multi-frame network processes sequential frames to estimate the structure of the static scene using planar-parallax geometry and the camera mounting position. Based on this reconstruction, it acts as a teacher, distilling knowledge such as scale information, masked drivable area, metric-scale depth for the static scene, and dynamic object mask to the singleframe network. It also aids the pose network in predicting a metric-scaled relative pose between two subsequent images. Our method achieved state-of-the-art results for the driving benchmark KITTI for metric-scaled depth prediction. Notably, it is one of the first methods to produce self-supervised metric-scaled depth prediction for the challenging Cityscapes dataset, demonstrating its effectiveness and versatility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。