分离解剖结构与语义信息,实现手术场景的精准4D重建。
DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting

- 将几何形变与语义演化独立建模,避免绑定导致的错位。
- 在CholecSeg8k和EndoVis18上达到最新性能,对组织变形鲁棒。
- 适合需要高精度语义-解剖对齐的智能手术系统研发者。
实时、文本提示驱动的4D重建对自主手术交互至关重要。现有方法因将视觉语言模型通过刚性耦合方式集成到可变形场中,导致语义与解剖结构严重错位。本文提出DeGenseGS——一种在4D高斯点阵中实现几何与语义解耦的手术场景理解框架。我们设计基于六边形平面的时空纠缠模块,利用共享运动隐变量同步语义变化与场景动态,同时显式分离语义更新与几何形变。为增强对重建伪影的鲁棒性,提出基于光栅化的语义提取机制,从拓扑连续的特征图中推断语义。此外,引入角对齐优化策略,符合原生超球面隐空间,防止语义失真。在CholecSeg8k和EndoVis18数据集上的大量实验表明,DeGenseGS实现最优性能,显著提升几何完整性与语义-解剖对齐度,即使面对剧烈组织形变与拓扑变化,仍能保持空间连续分割。
原文摘要 · Abstract (English)
Real-time, text-promptable 4D reconstruction is indispensable for autonomous surgical interaction. Severe misalignment between semantic meaning and physical anatomy still persists, largely because existing solutions integrate Vision-Language Models into deformable fields via a rigid coupling scheme that tightly binds semantic features to geometric warping. In this paper, we propose DeGenseGS, Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting, a novel framework that independently models semantic evolution and geometric deformation. Specifically, we propose a HexPlane-based spatiotemporal entanglement module that uses shared kinematic latents to synchronize semantic mutations with scene dynamics, while explicitly disentangling semantic updates from geometric deformation. To further ensure robustness against reconstruction artifacts, we devise a Rasterization-Native Semantic Extraction mechanism that infers semantics from topologically continuous feature maps. Additionally, we incorporate an angular-aligned optimization strategy that conforms to the native hyperspherical latent space, thereby preventing semantic distortion. Extensive evaluations on the CholecSeg8k and EndoVis18 datasets demonstrate that DeGenseGS achieves state-of-the-art performance. Our framework yields enhanced geometric completeness and robust semantic-anatomic alignment, enabling spatially continuous segmentation despite drastic tissue deformation and topological transitions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。