arXiv:2411.12510cs.CV2024-11被引 11

基于物理的可重光照3D重建,提升内窥镜图像真实感

PR-ENDO: Physically Based Relightable Gaussian Splatting for Endoscopy

  • 利用内窥镜光源与摄像头同轴特性,分离光照与组织属性
  • 引入专用MLP建模复杂光效,减少伪影并提升新视角泛化能力
  • 支持组织修改且保持物理光照一致性,贴近临床应用

内腔内窥镜检查是诊断消化道、泌尿生殖系统及呼吸道严重疾病的关键手段。从内窥镜图像实现3D重建与新视角合成,有望提升诊断效果。结合生理形变与内窥镜交互,可基于真实视频数据构建仿真工具。然而,受限的相机轨迹和视图依赖性光照导致重建出现伪影,易过拟合。本文提出PR-ENDO,一种新型3D重建框架,利用内窥镜成像中光源与摄像头高度对齐的独特性质,将光照效应与组织属性分离。该方法在传统3D高斯点云基础上引入基于物理的可重光照模型,通过专用MLP增强光传输建模,显著降低伪影并提升新视角泛化性能。在公开与自建数据集上,PR-ENDO均优于基线方法。相较于现有技术,其可在保持物理光照真实性的同时支持组织修改,更接近真实临床场景。

原文摘要 · Abstract (English)

Endoluminal endoscopic procedures are essential for diagnosing colorectal cancer and other severe conditions in the digestive tract, urogenital system, and airways. 3D reconstruction and novel-view synthesis from endoscopic images are promising tools for enhancing diagnosis. Moreover, integrating physiological deformations and interaction with the endoscope enables the development of simulation tools from real video data. However, constrained camera trajectories and view-dependent lighting create artifacts, leading to inaccurate or overfitted reconstructions. We present PR-ENDO, a novel 3D reconstruction framework leveraging the unique property of endoscopic imaging, where a single light source is closely aligned with the camera. Our method separates light effects from tissue properties. PR-ENDO enhances 3D Gaussian Splatting with a physically based relightable model. We boost the traditional light transport formulation with a specialized MLP capturing complex light-related effects while ensuring reduced artifacts and better generalization across novel views. PR-ENDO achieves superior reconstruction quality compared to baseline methods on both public and in-house datasets. Unlike existing approaches, PR-ENDO enables tissue modifications while preserving a physically accurate response to light, making it closer to real-world clinical use.

3D重建内窥镜可重光照高斯点云

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。