用双像素图像实现轻量级高精度深度估计,无需专用数据集
Revisiting Disparity from Dual-Pixel Images: Physics-Informed Lightweight Depth Estimation
- 基于补全网络显式约束视差,学习双像素物理特性
- 参数量仅为传统方法1/5,无DP数据集训练仍达顶尖性能
- 无需标注数据即可训练,适合嵌入式设备部署
本文提出一种基于双像素(DP)图像的高性能、轻量化视差(深度)估计方法。传统端到端深度学习方法参数量大,且未充分利用视差约束,限制了性能。为此,我们设计了一种基于补全的轻量网络,显式建模双像素特有的视差误差,并在训练中通过参数化采样引入该特性,使网络学习到双像素系统的物理与系统性视差规律。该方法可在无需双像素数据集的情况下,直接使用通用RGB-D数据集进行训练,避免了获取昂贵的双像素标注数据。此外,提出一种非学习型优化框架,通过修正网络输出的置信图有效缓解固有的视差扩展误差。实验表明,所提方法在不使用双像素数据集训练的前提下,仍达到当前最优性能,且系统规模仅为传统方法的1/5,验证了其高效性与实用性。代码与数据集已公开。
原文摘要 · Abstract (English)
In this study, we propose a high-performance disparity (depth) estimation method using dual-pixel (DP) images with few parameters. Conventional end-to-end deep-learning methods have many parameters but do not fully exploit disparity constraints, which limits their performance. Therefore, we propose a lightweight disparity estimation method based on a completion-based network that explicitly constrains disparity and learns the physical and systemic disparity properties of DP. By modeling the DP-specific disparity error parametrically and using it for sampling during training, the network acquires the unique properties of DP and enhances robustness. This learning also allows us to use a common RGB-D dataset for training without a DP dataset, which is labor-intensive to acquire. Furthermore, we propose a non-learning-based refinement framework that efficiently handles inherent disparity expansion errors by appropriately refining the confidence map of the network output. As a result, the proposed method achieved state-of-the-art results while reducing the overall system size to 1/5 of that of the conventional method, even without using the DP dataset for training, thereby demonstrating its effectiveness. The code and dataset are available on our project site.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。