无需训练的内窥镜血管运动放大框架,有效抑制误差累积并适应复杂手术场景。
EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Control
- 基于拉格朗日框架与周期性参考重置,分段更新参考帧防止误差积累。
- 双模组织感知放大:按运动或距离自适应调节放大强度,提升鲁棒性。
- 适用于含遮挡、器械干扰等复杂条件,适合临床手术辅助系统开发。
内窥镜手术中可视化细微血管运动对提高手术精度和决策至关重要,但受术中场景复杂多变影响仍具挑战。本文提出EndoControlMag,一种无需训练的拉格朗日框架,结合掩码控制的血管运动放大技术,专为内窥镜环境设计。核心包含两个模块:周期性参考重置(PRR)将视频分段为短重叠片段,动态更新参考帧以避免误差累积并保持时间连贯性;层级组织感知放大(HTM)采用双模式掩码膨胀策略,先通过预训练视觉追踪模型定位血管核心,再根据组织运动或距离实施自适应软化:运动驱动软化按组织位移调节放大强度,距离指数衰减模拟生物力学力衰减。该双模式可应对多样手术场景——运动软化适用于复杂组织变形,距离软化在光流不可靠时提供稳定性。我们在涵盖四种手术类型的EndoVMM24数据集上评估,包含遮挡、器械干扰、视角变化及血管形变等挑战场景。定量指标、视觉评估及专家外科医生评价均表明,EndoControlMag显著优于现有方法,在放大准确性和视觉质量方面表现突出,且跨复杂条件具有强鲁棒性。代码、数据集与视频结果见https://szupc.github.io/EndoControlMag/。
原文摘要 · Abstract (English)
Visualizing subtle vascular motions in endoscopic surgery is crucial for surgical precision and decision-making, yet remains challenging due to the complex and dynamic nature of surgical scenes. To address this, we introduce EndoControlMag, a training-free, Lagrangian-based framework with mask-conditioned vascular motion magnification tailored to endoscopic environments. Our approach features two key modules: a Periodic Reference Resetting (PRR) scheme that divides videos into short overlapping clips with dynamically updated reference frames to prevent error accumulation while maintaining temporal coherence, and a Hierarchical Tissue-aware Magnification (HTM) framework with dual-mode mask dilation. HTM first tracks vessel cores using a pretrained visual tracking model to maintain accurate localization despite occlusions and view changes. It then applies one of two adaptive softening strategies to surrounding tissues: motion-based softening that modulates magnification strength proportional to observed tissue displacement, or distance-based exponential decay that simulates biomechanical force attenuation. This dual-mode approach accommodates diverse surgical scenarios-motion-based softening excels with complex tissue deformations while distance-based softening provides stability during unreliable optical flow conditions. We evaluate EndoControlMag on our EndoVMM24 dataset spanning four different surgery types and various challenging scenarios, including occlusions, instrument disturbance, view changes, and vessel deformations. Quantitative metrics, visual assessments, and expert surgeon evaluations demonstrate that EndoControlMag significantly outperforms existing methods in both magnification accuracy and visual quality while maintaining robustness across challenging surgical conditions. The code, dataset, and video results are available at https://szupc.github.io/EndoControlMag/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。