arXiv:2506.04716cs.CV2025-06被引 7

用扩散模型学习专家内镜手术轨迹,提升预测精度与泛化能力。

Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion

  • 结合扩散模型与等变表示,建模专家操作的随机性与几何对称性。
  • 在近2000段视频上测试,轨迹预测准确率超越现有方法。
  • 适合做手术辅助系统研发或智能训练平台构建的研究者参考。

内镜黏膜下剥离术(ESD)是切除上皮病变的成熟技术。预测ESD视频中的剥离轨迹对提升外科技能培训和简化学习过程具有重要价值,但该领域仍研究不足。尽管模仿学习在从专家示范中获取技能方面展现出潜力,但在处理未来动作不确定性、学习几何对称性以及泛化到多样手术场景方面仍存在挑战。为此,我们提出一种新方法:基于等变表示的隐式扩散策略模仿学习(iDPOE)。该方法通过联合状态-动作分布建模专家行为,捕捉剥离轨迹的随机性,并实现跨多种内镜视角的鲁棒视觉表征学习。通过将扩散模型引入策略学习,iDPOE确保高效训练与采样,带来更精准预测与更好泛化性能。此外,通过嵌入等变性增强模型对几何对称性的泛化能力。为缓解状态不匹配问题,我们设计了前向过程引导的动作推断策略以支持条件采样。基于近2000个视频片段的数据集实验表明,本方法在轨迹预测上优于现有显式与隐式方法。据我们所知,这是首次将模仿学习应用于剥离轨迹预测的手术技能开发中。

原文摘要 · Abstract (English)

Endoscopic Submucosal Dissection (ESD) is a well-established technique for removing epithelial lesions. Predicting dissection trajectories in ESD videos offers significant potential for enhancing surgical skill training and simplifying the learning process, yet this area remains underexplored. While imitation learning has shown promise in acquiring skills from expert demonstrations, challenges persist in handling uncertain future movements, learning geometric symmetries, and generalizing to diverse surgical scenarios. To address these, we introduce a novel approach: Implicit Diffusion Policy with Equivariant Representations for Imitation Learning (iDPOE). Our method models expert behavior through a joint state action distribution, capturing the stochastic nature of dissection trajectories and enabling robust visual representation learning across various endoscopic views. By incorporating a diffusion model into policy learning, iDPOE ensures efficient training and sampling, leading to more accurate predictions and better generalization. Additionally, we enhance the model's ability to generalize to geometric symmetries by embedding equivariance into the learning process. To address state mismatches, we develop a forward-process guided action inference strategy for conditional sampling. Using an ESD video dataset of nearly 2000 clips, experimental results show that our approach surpasses state-of-the-art methods, both explicit and implicit, in trajectory prediction. To the best of our knowledge, this is the first application of imitation learning to surgical skill development for dissection trajectory prediction.

手术生成扩散模型模仿学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。