仅用首帧标注实现视频息肉分割,大幅降低人工成本。
First-frame Supervised Video Polyp Segmentation via Propagative and Semantic Dual-teacher Network
- 设计双教师网络,分别用跟踪和语义平均生成伪标签。
- 通过反向追踪评估伪标签质量,提升模型训练准确性。
- 在长视频上只需一帧标注,适合大规模医疗影像分析。
自动视频息肉分割在胃肠道癌症筛查中至关重要,但逐帧标注成本极高。虽然稀疏帧监督方法已降低标注负担,但在长视频和大规模数据集上仍不可行。本文首次将每段视频的标注成本降至仅需一帧,提出首帧监督视频息肉分割(FSVPS)任务,并设计传播与语义双教师网络(PSDNet)。该网络采用教师-学生框架,引入两类教师:传播教师为通用目标追踪器,将首帧标注传播至后续帧生成伪标签;语义教师为学生模型的指数移动平均,生成更稳定的时间不变伪标签。通过精心设计的反向传播策略融合两者伪标签,依据其反向追踪至首帧的空间一致性评估伪标签质量。在最大规模的VPS数据集SUN-SEG上,PSDNet性能媲美全监督方法,且相比稀疏帧监督最先进方法,Dice分数最低提升4.5%。
原文摘要 · Abstract (English)
Automatic video polyp segmentation plays a critical role in gastrointestinal cancer screening, but the cost of frameby-frame annotations is prohibitively high. While sparse-frame supervised methods have reduced this burden proportionately, the cost remains overwhelming for long-duration videos and large-scale datasets. In this paper, we, for the first time, reduce the annotation cost to just a single frame per polyp video, regardless of the video's length. To this end, we introduce a new task, First-Frame Supervised Video Polyp Segmentation (FSVPS), and propose a novel Propagative and Semantic Dual-Teacher Network (PSDNet). Specifically, PSDNet adopts a teacher-student framework but employs two distinct types of teachers: the propagative teacher and the semantic teacher. The propagative teacher is a universal object tracker that propagates the first-frame annotation to subsequent frames as pseudo labels. However, tracking errors may accumulate over time, gradually degrading the pseudo labels and misguiding the student model. To address this, we introduce the semantic teacher, an exponential moving average of the student model, which produces more stable and time-invariant pseudo labels. PSDNet merges the pseudo labels from both teachers using a carefully-designed back-propagation strategy. This strategy assesses the quality of the pseudo labels by tracking them backward to the first frame. High-quality pseudo labels are more likely to spatially align with the firstframe annotation after this backward tracking, ensuring more accurate teacher-to-student knowledge transfer and improved segmentation performance. Benchmarking on SUN-SEG, the largest VPS dataset, demonstrates the competitive performance of PSDNet compared to fully-supervised approaches, and its superiority over sparse-frame supervised state-of-the-arts with a minimum improvement of 4.5% in Dice score.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。