用两阶段方法提升无透镜相机成像的逼真度与一致性。
PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless Imaging
- 先用可变点扩散函数的去卷积法重建低频信息，保证数据一致
- 再用预训练扩散模型补全高频细节，提升图像真实感
- 适用于PhlatCam和DiffuserCam系统，适合图像重建研究者
无透镜相机相比传统带透镜系统在尺寸、重量和成本上具有显著优势。由于没有聚焦透镜，其依赖计算算法从多路复用测量中恢复场景。然而，现有算法因前向成像模型不准确且先验不足，难以重建高质量图像。为此，我们提出一种新的两阶段方法，实现一致且逼真的无透镜图像重建。第一阶段通过空间变化的去卷积方法，准确重建低频内容，适应视场内点扩散函数（PSF）的变化，确保数据一致性。第二阶段引入预训练扩散模型作为生成先验，以第一阶段恢复的低频内容为条件，有效重建通常在无透镜成像中丢失的高频细节，同时保持图像保真度。相比现有方法，本方法在数据保真与视觉质量之间取得更优平衡，已在两种主流无透镜系统PhlatCam和DiffuserCam上验证。项目网站：https://phocolens.github.io/。
原文摘要 · Abstract (English)
Lensless cameras offer significant advantages in size, weight, and cost compared to traditional lens-based systems. Without a focusing lens, lensless cameras rely on computational algorithms to recover the scenes from multiplexed measurements. However, current algorithms struggle with inaccurate forward imaging models and insufficient priors to reconstruct high-quality images. To overcome these limitations, we introduce a novel two-stage approach for consistent and photorealistic lensless image reconstruction. The first stage of our approach ensures data consistency by focusing on accurately reconstructing the low-frequency content with a spatially varying deconvolution method that adjusts to changes in the Point Spread Function (PSF) across the camera's field of view. The second stage enhances photorealism by incorporating a generative prior from pre-trained diffusion models. By conditioning on the low-frequency content retrieved in the first stage, the diffusion model effectively reconstructs the high-frequency details that are typically lost in the lensless imaging process, while also maintaining image fidelity. Our method achieves a superior balance between data fidelity and visual quality compared to existing methods, as demonstrated with two popular lensless systems, PhlatCam and DiffuserCam. Project website: https://phocolens.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。