利用双像素图像的离焦差异,通过多尺度相关性提升模糊图像还原效果。
Adjust Your Focus: Defocus Deblurring From Dual-Pixel Images Using Explicit Multi-Scale Cross-Correlation
- 显式计算双像素图像间的多尺度交叉相关,捕捉离焦差异信息
- 在多个数据集上优于现有方法,且计算量更低
- 适合需要高质量去模糊的摄影与计算机视觉应用
离焦模糊是摄影中的常见问题,通常由大光圈导致浅景深引起。虽然在人像模式中可能有意为之,但在其他场景下会影响视觉效果及下游任务如分割和深度估计。将模糊图像恢复为全聚焦图像是一个高度挑战且常为病态的问题。近期工作利用消费级单反相机和高端手机中常见的双像素(DP)图像信息来解决该问题。双像素传感器生成两个子孔径视图,包含离焦差异线索,其差异与焦点平面距离成正比。然而,现有方法仅采用通道拼接方式处理两个视图,未显式利用其中的差异信息。本文提出显式在两个双像素视图间进行交叉相关,引导网络在不同图像区域实现更精准的去模糊。我们采用多尺度交叉相关以处理不同尺度下的模糊与差异。定量和定性评估表明,所提多尺度交叉相关网络(MCCNet)在保持较低计算复杂度的同时,优于现有最先进方法。
原文摘要 · Abstract (English)
Defocus blur is a common problem in photography. It arises when an image is captured with a wide aperture, resulting in a shallow depth of field. Sometimes it is desired, e.g., in portrait effect. Otherwise, it is a problem from both an aesthetic point of view and downstream computer vision tasks, such as segmentation and depth estimation. Defocusing an out-of-focus image to obtain an all-in-focus image is a highly challenging and often ill-posed problem. A recent work exploited dual-pixel (DP) image information, widely available in consumer DSLRs and high-end smartphones, to solve the problem of defocus deblurring. DP sensors result in two sub-aperture views containing defocus disparity cues. A given pixel's disparity is directly proportional to the distance from the focal plane. However, the existing methods adopt a naïve approach of a channel-wise concatenation of the two DP views without explicitly utilizing the disparity cues within the network. In this work, we propose to perform an explicit cross-correlation between the two DP views to guide the network for appropriate deblurring in different image regions. We adopt multi-scale cross-correlation to handle blur and disparities at different scales. Quantitative and qualitative evaluation of our multi-scale cross-correlation network (MCCNet) reveals that it achieves better defocus deblurring than existing state-of-the-art methods despite having lesser computational complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。