提升内镜手术场景重建精度,解决颜色失真与形变建模难题。
ColorGS: High-fidelity Surgical Scene Reconstruction with Colored Gaussian Splatting
- 用可学习颜色的高斯点动态捕捉组织纹理变化。
- 实现局部形变与全局运动一致性的精准建模,PSNR达39.85。
- 适合需要高保真实时重建的手术导航与虚拟现实应用。
从内镜视频中高保真重建可变形组织仍具挑战,现有方法在捕捉细微颜色变化和建模全局形变方面存在局限。尽管3D高斯溅射(3DGS)能高效实现动态重建,但其固定的高斯体颜色分配难以应对复杂纹理,线性形变模型也无法建模一致的全局形变。为此,我们提出ColorGS框架,融合空间自适应颜色编码与增强形变建模。首先引入带可学习颜色参数的动态锚点高斯原语,显著提升复杂光照与组织相似性下的颜色表达能力;其次设计时间感知的高斯基函数结合可学习的时不变形变,精确捕捉局部组织形变与由手术交互引起的全局运动一致性。在达芬奇机器人手术视频及基准数据集(EndoNeRF、StereoMIS)上的实验表明,ColorGS达到当前最优性能,PSNR为39.85(比先前3DGS方法高1.5),SSIM达97.25%,同时保持实时渲染效率。本工作在高保真与计算实用性间取得平衡,对术中引导与AR/VR应用具有重要意义。
原文摘要 · Abstract (English)
High-fidelity reconstruction of deformable tissues from endoscopic videos remains challenging due to the limitations of existing methods in capturing subtle color variations and modeling global deformations. While 3D Gaussian Splatting (3DGS) enables efficient dynamic reconstruction, its fixed per-Gaussian color assignment struggles with intricate textures, and linear deformation modeling fails to model consistent global deformation. To address these issues, we propose ColorGS, a novel framework that integrates spatially adaptive color encoding and enhanced deformation modeling for surgical scene reconstruction. First, we introduce Colored Gaussian Primitives, which employ dynamic anchors with learnable color parameters to adaptively encode spatially varying textures, significantly improving color expressiveness under complex lighting and tissue similarity. Second, we design an Enhanced Deformation Model (EDM) that combines time-aware Gaussian basis functions with learnable time-independent deformations, enabling precise capture of both localized tissue deformations and global motion consistency caused by surgical interactions. Extensive experiments on DaVinci robotic surgery videos and benchmark datasets (EndoNeRF, StereoMIS) demonstrate that ColorGS achieves state-of-the-art performance, attaining a PSNR of 39.85 (1.5 higher than prior 3DGS-based methods) and superior SSIM (97.25\%) while maintaining real-time rendering efficiency. Our work advances surgical scene reconstruction by balancing high fidelity with computational practicality, critical for intraoperative guidance and AR/VR applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。