用单镜头+计算方法实现可调景深,效果媲美高端镜头。
Towards Single-Lens Controllable Depth-of-Field Imaging via Depth-Aware Point Spread Functions
- 基于深度感知的点扩散函数建模,修复镜头畸变。
- 在仿真数据集上实现多焦点图像重建,支持任意景深切换。
- 适合移动设备端轻量化景深控制,开源代码和数据集可用。
可控景深成像通常依赖昂贵的高端镜头,难以适配移动端需求。本文针对极简光学系统(MOS)存在的严重光学畸变与不可控景深问题,提出深度感知可控景深成像框架(DCDI)。该框架结合全聚焦畸变校正与单目深度估计,利用恢复后的图像和深度图,通过分块卷积生成任意高端镜头下的景深效果。为应对深度相关的退化问题,设计了深度感知退化自适应训练(DA2T)方案,并构建了基于不同物距下点扩散函数(PSF)仿真的深度感知畸变MOS(DAMOS)数据集。同时引入两个即插即用的深度感知机制,将深度信息融入畸变图像恢复。此外,提出一种存储高效的全景镜头场模型(Omni-Lens-Field),以表示多种镜头的4维PSF库。结合预测深度图、恢复图像与深度感知的PSF图,最终实现单镜头可控景深成像。实验表明,该方法显著提升重建质量,达到惊艳的可控景深效果,为该领域提供重要基准。源码与数据集将公开于https://github.com/XiaolongQian/DCDI。
原文摘要 · Abstract (English)
Controllable Depth-of-Field (DoF) imaging commonly produces amazing visual effects based on heavy and expensive high-end lenses. However, confronted with the increasing demand for mobile scenarios, it is desirable to achieve a lightweight solution with Minimalist Optical Systems (MOS). This work centers around two major limitations of MOS, i.e., the severe optical aberrations and uncontrollable DoF, for achieving single-lens controllable DoF imaging via computational methods. A Depth-aware Controllable DoF Imaging (DCDI) framework is proposed equipped with All-in-Focus (AiF) aberration correction and monocular depth estimation, where the recovered image and corresponding depth map are utilized to produce imaging results under diverse DoFs of any high-end lens via patch-wise convolution. To address the depth-varying optical degradation, we introduce a Depth-aware Degradation-adaptive Training (DA2T) scheme. At the dataset level, a Depth-aware Aberration MOS (DAMOS) dataset is established based on the simulation of Point Spread Functions (PSFs) under different object distances. Additionally, we design two plug-and-play depth-aware mechanisms to embed depth information into the aberration image recovery for better tackling depth-aware degradation. Furthermore, we propose a storage-efficient Omni-Lens-Field model to represent the 4D PSF library of various lenses. With the predicted depth map, recovered image, and depth-aware PSF map inferred by Omni-Lens-Field, single-lens controllable DoF imaging is achieved. Comprehensive experimental results demonstrate that the proposed framework enhances the recovery performance, and attains impressive single-lens controllable DoF imaging results, providing a seminal baseline for this field. The source code and the established dataset will be publicly available at https://github.com/XiaolongQian/DCDI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。