用四像素传感器提升手机摄像头深度估计精度
Disparity Estimation Using a Quad-Pixel Sensor
- 融合垂直与水平方向的相位差异信息进行深度推断
- 在合成数据集上训练，性能超越现有双像素和立体匹配方法
- 适合移动端深度感知、手机摄影与AR应用开发者
四像素（QP）传感器正越来越多地集成到商用手机摄像头中。每个单元由2×2个光电二极管共享一个微透镜组成，在失焦模糊时产生多方向相位偏移。类似双像素（DP）传感器，这种相位偏移可视为立体视差，用于深度估计。为此，我们提出一种四像素视差估计网络（QPDNet），通过融合垂直与水平方向的立体匹配相关性，有效利用丰富的QP信息实现精准视差估计。同时，我们设计了一种合成数据生成流程，基于现有RGB-Depth数据集构建训练数据。实验结果表明，QPDNet在多个指标上优于当前最先进的立体匹配与双像素方法。代码与合成数据集已开源：https://github.com/Zhuofeng-Wu/QPDNet。
原文摘要 · Abstract (English)
A quad-pixel (QP) sensor is increasingly integrated into commercial mobile cameras. The QP sensor has a unit of 2$\times$2 four photodiodes under a single microlens, generating multi-directional phase shifting when out-focus blurs occur. Similar to a dual-pixel (DP) sensor, the phase shifting can be regarded as stereo disparity and utilized for depth estimation. Based on this, we propose a QP disparity estimation network (QPDNet), which exploits abundant QP information by fusing vertical and horizontal stereo-matching correlations for effective disparity estimation. We also present a synthetic pipeline to generate a training dataset from an existing RGB-Depth dataset. Experimental results demonstrate that our QPDNet outperforms state-of-the-art stereo and DP methods. Our code and synthetic dataset are available at https://github.com/Zhuofeng-Wu/QPDNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。