融合多视角立体与物理成像模型,实现快速高精度水下场景重建。
Fast Underwater Scene Reconstruction using Multi-View Stereo and Physical Imaging
- 用MVS和物理成像模型双分支并行估计深度与介质参数。
- 在无真值深度条件下,训练速度提升3倍,渲染质量优于现有方法。
- 适合需要快速重建水下图像的科研与海洋探测应用。
水下场景重建因光与介质间的复杂相互作用(如散射与吸收)而极具挑战,导致深度估计与图像渲染困难。尽管基于神经辐射场(NeRF)的方法通过建模分离散射介质实现了高质量结果,但其训练与渲染速度仍较慢。为此,本文提出一种结合多视角立体(MVS)与物理驱动水下成像模型的新方法。该方法分为两个分支:一为基于传统成本体积管道的深度估计分支,另一为基于物理成像模型的渲染分支。深度分支优化场景几何,介质分支估计散射参数以实现精确渲染。与依赖真值深度的传统MVSNet不同,本方法无需地面真值深度,显著加快训练与渲染速度。通过介质子网络估计介质参数,并结合颜色MLP进行渲染,成功恢复真实水下色彩,获得更高保真度几何表示。实验表明,该方法在散射介质中生成新视角质量高,能清晰去除介质影响,且在渲染质量与训练效率上均优于现有方法。
原文摘要 · Abstract (English)
Underwater scene reconstruction poses a substantial challenge because of the intricate interplay between light and the medium, resulting in scattering and absorption effects that make both depth estimation and rendering more complex. While recent Neural Radiance Fields (NeRF) based methods for underwater scenes achieve high-quality results by modeling and separating the scattering medium, they still suffer from slow training and rendering speeds. To address these limitations, we propose a novel method that integrates Multi-View Stereo (MVS) with a physics-based underwater image formation model. Our approach consists of two branches: one for depth estimation using the traditional cost volume pipeline of MVS, and the other for rendering based on the physics-based image formation model. The depth branch improves scene geometry, while the medium branch determines the scattering parameters to achieve precise scene rendering. Unlike traditional MVSNet methods that rely on ground-truth depth, our method does not necessitate the use of depth truth, thus allowing for expedited training and rendering processes. By leveraging the medium subnet to estimate the medium parameters and combining this with a color MLP for rendering, we restore the true colors of underwater scenes and achieve higher-fidelity geometric representations. Experimental results show that our method enables high-quality synthesis of novel views in scattering media, clear views restoration by removing the medium, and outperforms existing methods in rendering quality and training efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。