用波动特征与光流约束,提升内窥镜视频3D重建精度
EndoWave: Rational-Wavelet 4D Gaussian Splatting for Endoscopic Reconstruction
- 4D高斯点云统一建模,结合光流几何约束增强时间一致性
- 多分辨率有理正交小波分离镜头细节,提升渲染质量
- 在真实手术数据集上超越现有方法,适合微创手术视觉重建
在机器人辅助微创手术中,从内窥镜视频准确重建3D结构对下游任务和治疗效果至关重要。然而,内窥镜场景存在光照不一致、组织非刚性运动及视角依赖的反光等挑战。基于3DGS的方法仅依赖外观约束进行优化,常因动态视觉伪影误导优化过程,导致重建失真。为此,本文提出EndoWave,一种融合光流几何约束与多分辨率有理小波监督的统一时空高斯点云框架。首先,采用直接在4D域优化的时空高斯表示;其次,利用光流推导几何约束以增强时间连贯性并有效约束场景结构;最后,引入多分辨率有理正交小波作为约束,有效分离内窥镜细节,提升渲染性能。在两个真实手术数据集EndoNeRF和StereoMIS上的大量实验表明,本方法在重建质量和视觉准确性上均优于基线方法。
原文摘要 · Abstract (English)
In robot-assisted minimally invasive surgery, accurate 3D reconstruction from endoscopic video is vital for downstream tasks and improved outcomes. However, endoscopic scenarios present unique challenges, including photometric inconsistencies, non-rigid tissue motion, and view-dependent highlights. Most 3DGS-based methods that rely solely on appearance constraints for optimizing 3DGS are often insufficient in this context, as these dynamic visual artifacts can mislead the optimization process and lead to inaccurate reconstructions. To address these limitations, we present EndoWave, a unified spatio-temporal Gaussian Splatting framework by incorporating an optical flow-based geometric constraint and a multi-resolution rational wavelet supervision. First, we adopt a unified spatio-temporal Gaussian representation that directly optimizes primitives in a 4D domain. Second, we propose a geometric constraint derived from optical flow to enhance temporal coherence and effectively constrain the 3D structure of the scene. Third, we propose a multi-resolution rational orthogonal wavelet as a constraint, which can effectively separate the details of the endoscope and enhance the rendering performance. Extensive evaluations on two real surgical datasets, EndoNeRF and StereoMIS, demonstrate that our method EndoWave achieves state-of-the-art reconstruction quality and visual accuracy compared to the baseline method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。