基于内窥镜稀疏视角,用不确定性感知提升手术场景的三维重建质量
UC-NeRF: Uncertainty-aware Conditional Neural Radiance Fields from Endoscopic Sparse Views
- 引入多视图不确定性估计,动态调节神经辐射场以应对光照不一致
- 在SCARED和Hamlyn数据集上,外观与几何重建精度均超越当前最优方法
- 适合需要高保真手术可视化、尤其是稀疏视角场景的研究者使用
术中场景可视化对揭示微创手术中的内部解剖结构至关重要。新视角合成技术能重建几何与外观,提升术前规划与决策能力。尽管神经辐射场(NeRF)表现优异,但其直接应用于手术场景时受限于内窥镜稀疏视角和显著的光度不一致性,导致重建效果不佳。本文提出不确定性感知的条件神经辐射场(UC-NeRF),以缓解稀疏视图下的形状-辐射模糊问题。核心思想是将多视图不确定性估计作为条件,自适应建模光度不一致性。具体地,先通过多视图立体网络建立稀疏视图间的几何对应关系,生成不确定性估计与特征先验;在神经渲染中,设计基线自适应的NeRF网络,利用不确定性估计显式处理光照不一致;此外,引入不确定性引导的几何蒸馏机制强化几何学习。在SCARED和Hamlyn数据集上的实验表明,本方法在外观与几何重建方面均显著优于现有最先进方法。代码将开源至https://github.com/wrld/UC-NeRF。
原文摘要 · Abstract (English)
Visualizing surgical scenes is crucial for revealing internal anatomical structures during minimally invasive procedures. Novel View Synthesis is a vital technique that offers geometry and appearance reconstruction, enhancing understanding, planning, and decision-making in surgical scenes. Despite the impressive achievements of Neural Radiance Field (NeRF), its direct application to surgical scenes produces unsatisfying results due to two challenges: endoscopic sparse views and significant photometric inconsistencies. In this paper, we propose uncertainty-aware conditional NeRF for novel view synthesis to tackle the severe shape-radiance ambiguity from sparse surgical views. The core of UC-NeRF is to incorporate the multi-view uncertainty estimation to condition the neural radiance field for modeling the severe photometric inconsistencies adaptively. Specifically, our UC-NeRF first builds a consistency learner in the form of multi-view stereo network, to establish the geometric correspondence from sparse views and generate uncertainty estimation and feature priors. In neural rendering, we design a base-adaptive NeRF network to exploit the uncertainty estimation for explicitly handling the photometric inconsistencies. Furthermore, an uncertainty-guided geometry distillation is employed to enhance geometry learning. Experiments on the SCARED and Hamlyn datasets demonstrate our superior performance in rendering appearance and geometry, consistently outperforming the current state-of-the-art approaches. Our code will be released at https://github.com/wrld/UC-NeRF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。