无需相机姿态信息,也能实现精准单目深度估计自适应。
No Pose Estimation? No Problem: Pose-Agnostic and Instance-Aware Test-Time Adaptation for Monocular Depth Estimation

- 不依赖相机姿态,通过实例感知掩码提升适应能力。
- 在DrivingStereo和Waymo数据集上显著超越现有方法。
- 适合部署于动态变化环境中的实时深度估计系统。
单目深度估计(MDE)从单张RGB图像中推断像素级深度,在需三维场景理解的AI应用中至关重要。现实场景中,模型常面临与训练环境不同的条件,测试时域自适应(TTA)是有效应对策略。尽管已有自监督TTA方法取得进展,但在多变动态环境中仍表现不佳。为此,本文提出新型高效TTA框架PITTA,包含两大创新:(i) 无姿态依赖的TTA范式,无需相机姿态信息即可实现有效适配;(ii) 实例感知图像掩码,利用预训练全景分割网络生成掩码,剔除静态背景后提取动态物体(如车辆、行人)的实例掩码。此外,引入简单有效的边缘提取方法优化输入图像与深度图。在DrivingStereo与Waymo数据集上,不同环境条件下实验表明,PITTA显著优于现有SOTA方法。
原文摘要 · Abstract (English)
Monocular depth estimation (MDE), inferring pixel-level depths in single RGB images from a monocular camera, plays a crucial and pivotal role in a variety of AI applications demanding a three-dimensional (3D) topographical scene. In the real-world scenarios, MDE models often need to be deployed in environments with different conditions from those for training. Test-time (domain) adaptation (TTA) is one of the compelling and practical approaches to address the issue. Although there have been notable advancements in TTA for MDE, particularly in a self-supervised manner, existing methods are still ineffective and problematic when applied to diverse and dynamic environments. To break through this challenge, we propose a novel and high-performing TTA framework for MDE, named PITTA. Our approach incorporates two key innovative strategies: (i) pose-agnostic TTA paradigm for MDE and (ii) instance-aware image masking. Specifically, PITTA enables highly effective TTA on a pretrained MDE network in a pose-agnostic manner without resorting to any camera pose information. Besides, our instance-aware masking strategy extracts instance-wise masks for dynamic objects (e.g., vehicles, pedestrians, etc.) from a segmentation mask produced by a pretrained panoptic segmentation network, by removing static objects including background components. To further boost performance, we also present a simple yet effective edge extraction methodology for the input image (i.e., a single monocular image) and depth map. Extensive experimental evaluations on DrivingStereo and Waymo datasets with varying environmental conditions demonstrate that our proposed framework, PITTA, surpasses the existing state-of-the-art techniques with remarkable performance improvements in MDE during TTA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。