arXiv:2607.14968cs.CV2026-07

用全景拼接扩展内窥镜视野,实时提升分割与追踪精度

Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction

论文配图:Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction
图 1 · 摘自论文原文
  • 构建显式全景画布,实时拼接多帧有效观测
  • 在多个基准上提升分割和追踪性能,保持60+ FPS
  • 无需重训练现有模型,适合手术室实时应用

外科视频理解是导航系统的基础。内窥镜感知常受限于视野狭窄和器械遮挡,时空上下文对鲁棒推断至关重要。现有视频模型通常通过学习特征表示隐式存储历史信息,但需特定任务训练、大量标注数据且计算开销大。本文提出Stitch-Inferencer,一种实时、模型无关的推理框架,将隐式特征记忆替换为显式图像空间全景画布。通过跨帧拼接有效观测,该框架在在线无器械视图中保留先前观察到的像素,扩大有效视野,并直接访问当前帧缺失或暂时被遮挡的区域。下游分割或跟踪模型在全景图的紧凑兴趣区上运行,预测结果再投影回当前帧,使现有模型无需重训练即可利用长程上下文。在解剖结构分割和点/框追踪任务上的实验表明,该方法在多种基线模型上均实现一致提升,同时保持实时吞吐量。仅拼接模块即达60+ FPS,为计算受限的术中环境提供实用的推理时增强方案。源代码将公开。

原文摘要 · Abstract (English)

Surgical video understanding is fundamental to navigation systems. Endoscopic perception is often hindered by a limited field-of-view and frequent instrument occlusions, making spatio-temporal context essential for robust inference. These challenges have motivated video models that aggregate information across frames. However, existing video models typically store past observations implicitly in learned feature representations, often requiring task-specific video training, substantial annotated data, and increased computational cost. We propose Stitch-Inferencer, a real-time, model-agnostic inference framework that replaces implicit feature memory with an explicit image-space panoramic canvas. By stitching valid observations across frames, Stitch-Inferencer preserves previously observed pixels in an online, instrument-free view, expanding the effective field-of-view and providing direct access to regions that are temporarily occluded or absent from the current frame. Downstream segmentation or tracking models are applied to a compact region of interest on the panorama, and their predictions are reprojected to the current frame, enabling existing models to exploit long-range context without retraining. Experiments on anatomy segmentation and point/box tracking demonstrate consistent improvements across diverse baselines while preserving real-time throughput. The stitching module alone runs at over 60 FPS, providing a practical inference-time solution to enhance endoscopic perception in computationally constrained intraoperative environments. Source code will be made publicly available.

内窥镜视频分割全景重建实时推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。