结合大模型与高斯溅射，实现完整内窥镜场景重建。
EndoLRMGS: Complete Endoscopic Scene Reconstruction combining Large Reconstruction Modelling and Gaussian Splatting
- 用大重建模型和高斯溅射融合建模手术器械与可变形组织。
- 工具3D投影交并比提升超40%，渲染质量指标显著提高。
- 适合需要精准三维重建的机器人辅助手术研究者。
完整的手术场景重建对机器人辅助手术至关重要。深度估计虽有潜力，但现有方法在深度不连续处易产生噪声，且无法完整重建被遮挡表面。为此，我们提出EndoLRMGS，结合大重建模型（LRM）与高斯溅射（GS），实现完整手术场景重建。GS用于重建可变形组织，LRM生成手术器械的3D模型，通过引入正交透视联合投影优化（OPjPO）对位置与尺度进行优化，提升精度。在三个公开数据集的四段手术视频上实验显示，本方法使工具3D模型在2D投影中的交并比（IoU）提升超过40%；工具投影的PSNR从3.82%增至11.07%；组织渲染质量也显著改善，PSNR从0.46%升至49.87%，SSIM从1.53%增至29.21%。
原文摘要 · Abstract (English)
Complete reconstruction of surgical scenes is crucial for robot-assisted surgery (RAS). Deep depth estimation is promising but existing works struggle with depth discontinuities, resulting in noisy predictions at object boundaries and do not achieve complete reconstruction omitting occluded surfaces. To address these issues we propose EndoLRMGS, that combines Large Reconstruction Modelling (LRM) and Gaussian Splatting (GS), for complete surgical scene reconstruction. GS reconstructs deformable tissues and LRM generates 3D models for surgical tools while position and scale are subsequently optimized by introducing orthogonal perspective joint projection optimization (OPjPO) to enhance accuracy. In experiments on four surgical videos from three public datasets, our method improves the Intersection-over-union (IoU) of tool 3D models in 2D projections by>40%. Additionally, EndoLRMGS improves the PSNR of the tools projection from 3.82% to 11.07%. Tissue rendering quality also improves, with PSNR increasing from 0.46% to 49.87%, and SSIM from 1.53% to 29.21% across all test videos.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。