无需固定视角,也能精准重建动态手术场景的4D影像。
4D Monocular Surgical Reconstruction under Arbitrary Camera Motions
- 分窗口渐进建模,适应任意运动的单目内窥镜视频。
- 在三个公开数据集上,外观与几何精度均超越现有方法。
- 适合临床实时重建,对初始化要求低,稳定性强。
从内窥镜视频中重建可变形手术场景具有挑战性且临床意义重大。现有基于隐式神经表示或3D高斯溅射的方法虽有进展,但多针对固定视角的场景,依赖立体深度先验或精确的运动结构(SfM)进行初始化与优化,难以处理真实临床中存在大范围相机运动的单目序列。为此,我们提出Local-EndoGS,一种适用于任意相机运动下单目内窥镜序列的高质量4D重建框架。该方法采用逐阶段、基于窗口的全局表示,为每个观测窗口分配局部可变形场景模型,实现对长序列和大幅运动的可扩展建模。为克服缺乏立体深度或精确SfM时的不可靠初始化问题,设计了融合多视图几何、跨窗口信息与单目深度先验的粗到精策略,为优化提供鲁棒基础。进一步引入长程2D像素轨迹约束与物理运动先验,提升形变合理性。在三个包含可变形场景与不同相机运动的公开数据集上的实验表明,Local-EndoGS在外观质量和几何精度上持续优于现有先进方法。消融实验验证了关键设计的有效性。代码将在录用后公开:https://github.com/IRMVLab/Local-EndoGS。
原文摘要 · Abstract (English)
Reconstructing deformable surgical scenes from endoscopic videos is challenging and clinically important. Recent state-of-the-art methods based on implicit neural representations or 3D Gaussian splatting have made notable progress. However, most are designed for deformable scenes with fixed endoscope viewpoints and rely on stereo depth priors or accurate structure-from-motion for initialization and optimization, limiting their ability to handle monocular sequences with large camera motion in real clinical settings. To address this, we propose Local-EndoGS, a high-quality 4D reconstruction framework for monocular endoscopic sequences with arbitrary camera motion. Local-EndoGS introduces a progressive, window-based global representation that allocates local deformable scene models to each observed window, enabling scalability to long sequences with substantial motion. To overcome unreliable initialization without stereo depth or accurate structure-from-motion, we design a coarse-to-fine strategy integrating multi-view geometry, cross-window information, and monocular depth priors, providing a robust foundation for optimization. We further incorporate long-range 2D pixel trajectory constraints and physical motion priors to improve deformation plausibility. Experiments on three public endoscopic datasets with deformable scenes and varying camera motions show that Local-EndoGS consistently outperforms state-of-the-art methods in appearance quality and geometry. Ablation studies validate the effectiveness of our key designs. Code will be released upon acceptance at: https://github.com/IRMVLab/Local-EndoGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。