用零训练模块让SAM2稳定追踪肠镜视频中的息肉，兼顾精度与泛化能力。
FreeVPS: Repurposing Training-Free SAM2 for Generalizable Video Polyp Segmentation
- 将息肉分割转为检测跟踪范式，利用SAM2的时空建模能力。
- 在真实肠镜视频中实现领先性能，跨域测试准确率超90%。
- 适合临床场景部署，无需微调即可应对长视频跟踪挑战。
现有视频息肉分割方法难以平衡时空建模与域泛化能力，限制其临床应用。本文提出FreeVPS，将视频息肉分割重构为检测-跟踪范式，借助图像息肉分割模型捕捉空间上下文，并融合段一切模型2（SAM2）的时序建模能力。然而，长时间肠镜视频中SAM2存在误差累积问题，导致分割不稳定。为此，我们引入两个无训练模块：内部关联过滤模块消除检测阶段的空间误判，降低假阳性；外部关联优化模块自适应更新记忆库，防止误差传播，增强时序一致性。两者协同作用使SAM2实现稳定分割，在域内与域外场景均达到顶尖水平。实验表明，FreeVPS可在长达数分钟的未修剪肠镜视频中保持鲁棒跟踪，具备实际临床分析潜力。
原文摘要 · Abstract (English)
Existing video polyp segmentation (VPS) paradigms usually struggle to balance between spatiotemporal modeling and domain generalization, limiting their applicability in real clinical scenarios. To embrace this challenge, we recast the VPS task as a track-by-detect paradigm that leverages the spatial contexts captured by the image polyp segmentation (IPS) model while integrating the temporal modeling capabilities of segment anything model 2 (SAM2). However, during long-term polyp tracking in colonoscopy videos, SAM2 suffers from error accumulation, resulting in a snowball effect that compromises segmentation stability. We mitigate this issue by repurposing SAM2 as a video polyp segmenter with two training-free modules. In particular, the intra-association filtering module eliminates spatial inaccuracies originating from the detecting stage, reducing false positives. The inter-association refinement module adaptively updates the memory bank to prevent error propagation over time, enhancing temporal coherence. Both modules work synergistically to stabilize SAM2, achieving cutting-edge performance in both in-domain and out-of-domain scenarios. Furthermore, we demonstrate the robust tracking capabilities of FreeVPS in long-untrimmed colonoscopy videos, underscoring its potential reliable clinical analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。