用扩散模型重建三维云体，让单视角图像更逼真。
Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes
- 用扩散模型学习体积场分布，基于新数据集训练
- 实现单视角下云体重建，质量远超以往
- 适合做三维视觉、生成模型的研究者
我们提出一种单视角体积场重建方法，适用于存在多重散射的场景，如云层。通过在包含1000个合成体积密度场的新基准数据集上训练无条件扩散模型，学习未知体积场分布。采用新型可扩散的单平面表示法，在潜在空间中训练神经扩散模型。将定制化的参数化扩散后验采样技术融入不同重建任务，并使用基于物理的可微分体积渲染器，为潜在空间中的光传输提供梯度。该方法突破传统NeRF范式，使重建结果更贴合观测数据。实验表明，该方法实现了前所未有的单视角云体重建质量。
原文摘要 · Abstract (English)
We introduce a single-view reconstruction technique of volumetric fields in which multiple light scattering effects are omnipresent, such as in clouds. We model the unknown distribution of volumetric fields using an unconditional diffusion model trained on a novel benchmark dataset comprising 1,000 synthetically simulated volumetric density fields. The neural diffusion model is trained on the latent codes of a novel, diffusion-friendly, monoplanar representation. The generative model is used to incorporate a tailored parametric diffusion posterior sampling technique into different reconstruction tasks. A physically-based differentiable volume renderer is employed to provide gradients with respect to light transport in the latent space. This stands in contrast to classic NeRF approaches and makes the reconstructions better aligned with observed data. Through various experiments, we demonstrate single-view reconstruction of volumetric clouds at a previously unattainable quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。