手术视频实时重建3D器官,无需精确轨迹先验
Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery

- 在线联合优化相机位姿与可变形场景表示
- 在无或噪声轨迹先验下仍保持高精度重建
- 适合机器人手术中实时三维可视化
高斯点阵是当前机器人辅助微创手术(RAMIS)中密集可变形3D解剖结构重建的最先进方法;然而,多数流程为离线处理,依赖准确的相机轨迹先验(通常来自机器人运动学),当先验缺失或有噪声时应用受限。为此,我们提出Track2Map,一种从手术视频直接在线联合优化相机轨迹与3D可变形场景表示的高斯点阵流水线。该方法可在无或噪声轨迹先验情况下实现鲁棒3D重建,并因其在线特性有效作为同步定位与地图构建(SLAM)方法。为应对组织运动和模糊视觉线索带来的优化不稳定性,我们引入基于密集2D点跟踪的轨迹锚定形变初始化,并利用轨迹统计量分离相机运动与场景形变,通过检测静止相机时段减少增量映射中的漂移。在StereoMIS数据集上的实验表明,Track2Map在重建质量与相机轨迹精度上均优于现有SLAM方法,也优于依赖轨迹先验的非SLAM方法。代码已开源:https://track2map.github.io/
原文摘要 · Abstract (English)
Gaussian splatting is the current state-of-the-art for dense, deformable 3D anatomy reconstruction in robot-assisted minimally invasive surgery (RAMIS); however, most pipelines are offline and depend on accurate camera trajectory priors (often from robotic kinematics), limiting applicability when priors are missing or noisy. To address these limitations, we propose Track2Map, an online 3D Gaussian Splatting pipeline that jointly optimizes camera trajectory and 3D deformable scene representation directly from surgical video. Track2Map is therefore capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy, and due to its online nature it effectively works as a Simultaneous Localisation and Mapping (SLAM) method. To stabilize optimization in the presence of tissue motion and ambiguous visual cues, we introduce a track-anchored deformation initialization using dense 2D point tracks. Track statistics are further utilized to disentangle camera motion from scene deformation by detecting static camera periods and reducing drift during incremental mapping. Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods, as well as compared to non-SLAM methods that utilize camera trajectory priors. The code is available at https://track2map.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。