arXiv:2504.16636cs.CV2025-04TPAMI被引 3

用双摄手机数据重建全焦深神经辐射场,无需手动对焦。

Dual-Camera All-in-Focus Neural Radiance Fields

论文配图:Dual-Camera All-in-Focus Neural Radiance Fields
图 1 · 摘自论文原文
  • 采用主摄与超广角双摄像头,利用后者大景深作为参考恢复清晰图像。
  • 提出可学习模糊参数的融合模块,有效还原全焦深视角。
  • 适合移动端3D成像、虚拟拍摄等需要自动对焦的应用场景。

我们提出首个无需人工对焦即可合成全焦深神经辐射场(NeRF)的框架。传统单相机方法因持续模糊和缺乏清晰参照而失效。本工作利用智能手机双摄系统:主摄高分辨率,超广角具有更大景深(DoF)。通过空间扭曲与色彩匹配对齐双摄图像,再设计基于模糊感知的融合模块,学习模糊参数并预测模糊图以融合图像。构建了包含主摄与超广角配对图像的多视角数据集。大量实验表明,所提方法DC-NeRF能生成高质量全焦深新视角,在定量与定性指标上优于强基线。还可实现可调模糊强度与焦点平面,支持对焦切换与分光镜效果。

原文摘要 · Abstract (English)

We present the first framework capable of synthesizing the all-in-focus neural radiance field (NeRF) from inputs without manual refocusing. Without refocusing, the camera will automatically focus on the fixed object for all views, and current NeRF methods typically using one camera fail due to the consistent defocus blur and a lack of sharp reference. To restore the all-in-focus NeRF, we introduce the dual-camera from smartphones, where the ultra-wide camera has a wider depth-of-field (DoF) and the main camera possesses a higher resolution. The dual camera pair saves the high-fidelity details from the main camera and uses the ultra-wide camera's deep DoF as reference for all-in-focus restoration. To this end, we first implement spatial warping and color matching to align the dual camera, followed by a defocus-aware fusion module with learnable defocus parameters to predict a defocus map and fuse the aligned camera pair. We also build a multi-view dataset that includes image pairs of the main and ultra-wide cameras in a smartphone. Extensive experiments on this dataset verify that our solution, termed DC-NeRF, can produce high-quality all-in-focus novel views and compares favorably against strong baselines quantitatively and qualitatively. We further show DoF applications of DC-NeRF with adjustable blur intensity and focal plane, including refocusing and split diopter.

NeRF双摄全焦深手机摄影

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。