arXiv:2609.02717cs.CVcs.RO2026-09

首个多视角手术感知基准,用于评估内窥镜3D重建效果。

MV-dVRK: A Multi-Viewpoint Benchmark for Spatial Surgical Perception

论文配图:MV-dVRK: A Multi-Viewpoint Benchmark for Spatial Surgical Perception
图 1 · 摘自论文原文
  • 构建多视角同步双目图像与精确几何数据的手术数据集
  • 三视角下优化方法覆盖67%真实表面点,误差小于1毫米
  • 适合研究手术机器人视觉、3D重建与多视角感知的学者

大规模训练和精细优化显著提升了稀疏多视角3D重建性能。然而,这些方法尚未在真实的内窥镜图像上进行严格评估。当前临床远程手术机器人仅使用单个立体摄像头,导致多视角数据极为罕见。本文提出MV-dVRK,首个体外手术数据集,结合多个曝光同步的立体视角、精确表面几何与相机位姿。静态子集提供密集SfM参考几何,经工业3D扫描验证,并包含真值相机位姿与稀疏视角测试集。我们利用MV-dVRK系统比较零样本单目、双目、多双目及多视角3D重建方法随视角数增加的表现。使用两个内窥镜时,多双目重建覆盖最优;引入第三个视角后,基于优化的多视角方法表现最佳,在1毫米容差内覆盖67%的真实表面点,并恢复高精度相对相机位姿。相比之下,前馈基础模型在同一设置下仅覆盖43%的真实表面。数据集还包含十段动态序列,涵盖多种手术任务,具有递增的运动复杂性和组织形变,为未来多视角手术感知研究奠定基础。项目地址:https://mv-dvrk.is.mpg.de。

原文摘要 · Abstract (English)

Large-scale training and refined optimization techniques have greatly improved sparse multi-view 3D reconstruction. Despite their relevance to surgery, such methods have never before been rigorously evaluated on real endoscopic images. Current clinical telerobots deploy a single stereo camera inside the patient, making multi-viewpoint data extremely rare. This paper presents MV-dVRK, the first ex-vivo surgical dataset to combine multiple exposure-synchronized stereo viewpoints with accurate surface geometry and camera poses. The static subset of the benchmark provides dense SfM reference geometry, validated against an industrial 3D scanner, together with ground-truth camera poses and sparse-view test sets. We use MV-dVRK to systematically compare zero-shot monocular, stereo, multi-stereo, and multi-view 3D reconstruction methods as the number of viewpoints increases. With two endoscopes, multi-stereo reconstruction achieves the highest coverage. With a third viewpoint, optimization-based multi-view methods perform best, covering 67% of ground-truth surface points within a 1 mm tolerance and recovering highly accurate relative camera poses. By contrast, feed-forward foundation models cover only 43% of the ground-truth surface in the same setting. MV-dVRK also includes ten dynamic sequences spanning multiple surgical tasks, with increasing kinematic complexity and tissue deformation, providing a basis for future research in multi-viewpoint surgical perception. The project is available at: https://mv-dvrk.is.mpg.de.

手术感知3D重建多视角数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。