用扩散模型增强内窥镜视角外推,减少重建伪影。
ExtraGS: Enhancing Endoscopic View Extrapolation via Diffusion-Guided 3D Gaussian Splatting

- 通过不确定性引导采样主动探索盲区
- 用扩散模型生成伪观测修复结构,降低伪影
- 适合机器人微创手术视觉感知研究者
机器人辅助微创手术(MIS)依赖可靠的内窥镜感知进行导航与安全。然而,传统内窥镜视野有限,大量周围解剖结构无法观察。近年来的神经渲染方法如神经辐射场和3D高斯泼溅可从内窥镜视频合成新视角,但依赖稀疏观测时,沿训练轨迹外推常产生严重伪影。本文提出ExtraGS框架,通过扩散引导的3D高斯泼溅增强内窥镜视角外推。从初始重建出发,引入不确定性引导的虚拟相机采样策略,主动探索盲区并最大化信息增益。在这些采样位置生成的视图通过扩散模型优化,恢复合理解剖结构,形成伪观测以指导后续优化。为防止生成内容破坏可靠区域,采用置信度加权微调策略融合伪观测。在多个公开内窥镜数据集上的实验表明,ExtraGS显著降低外推伪影,实现当前最优的内窥镜新视角合成性能。
原文摘要 · Abstract (English)
Robot-assisted minimally invasive surgery (MIS) critically depends on reliable endoscopic perception for navigation and safety. However, conventional endoscopes provide only a limited field of view, leaving large portions of the surrounding anatomy unobserved. Recent neural rendering approaches, such as Neural Radiance Fields and 3D Gaussian Splatting, enable novel view synthesis from endoscopic videos, but their reliance on sparse observations often leads to severe artifacts when extrapolating beyond the training trajectory. In this work, we propose ExtraGS, a framework for enhancing endoscopic view extrapolation through diffusion-guided 3D Gaussian Splatting. Starting from an initial reconstruction, we introduce an uncertainty-guided virtual camera sampling strategy to actively explore blind spots and maximize information gain. The rendered views from these sampled locations are refined using a diffusion model to recover plausible anatomical structures, producing pseudo-observations that guide further optimization. To prevent the generated content from degrading reliable regions, we adopt a confidence-weighted fine-tuning strategy when incorporating these pseudo-observations. Extensive experiments on multiple public endoscopic datasets demonstrate that ExtraGS significantly reduces extrapolation artifacts and achieves state-of-the-art performance in endoscopic novel view synthesis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。