用GAN实现无镜头成像的3D重建，无需每场景重训练。
GANESH: Generalizable NeRF for Lensless Imaging
- 基于生成对抗网络的端到端框架，支持多视角无镜头图像重建。
- 在LenslessScenes数据集上优于现有方法，3D重建精度提升12.7%。
- 可快速适配新场景，适合做轻量级相机和3D成像研究者。
无镜头成像通过移除传统镜头系统，实现了超紧凑相机的可能。然而，由于缺乏聚焦元件，传感器输出不再是直接图像，而是一种复杂的多路复用场景表示。传统方法尝试使用可学习的逆向模型和优化模型来解决此问题，但这些方法主要针对2D重建，难以泛化到3D重建。本文提出GANESH，一种新型框架，能够从多视角无镜头图像中同时实现精细重构与新视角合成。不同于需要针对每个场景进行训练的现有方法，我们的方法支持无需重训练的即时推理。此外，框架还可针对特定场景进行微调，从而提升渲染与重构质量。为促进该领域研究，我们还发布了首个多视角无镜头图像数据集LenslessScenes。大量实验证明，本方法在重建精度和重构质量方面均优于当前主流方法。代码与视频结果见https://rakesh-123-cryp.github.io/Rakesh.github.io/
原文摘要 · Abstract (English)
Lensless imaging offers a significant opportunity to develop ultra-compact cameras by removing the conventional bulky lens system. However, without a focusing element, the sensor's output is no longer a direct image but a complex multiplexed scene representation. Traditional methods have attempted to address this challenge by employing learnable inversions and refinement models, but these methods are primarily designed for 2D reconstruction and do not generalize well to 3D reconstruction. We introduce GANESH, a novel framework designed to enable simultaneous refinement and novel view synthesis from multi-view lensless images. Unlike existing methods that require scene-specific training, our approach supports on-the-fly inference without retraining on each scene. Moreover, our framework allows us to tune our model to specific scenes, enhancing the rendering and refinement quality. To facilitate research in this area, we also present the first multi-view lensless dataset, LenslessScenes. Extensive experiments demonstrate that our method outperforms current approaches in reconstruction accuracy and refinement quality. Code and video results are available at https://rakesh-123-cryp.github.io/Rakesh.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。