用时间结构自监督学习，让模型从27段结肠镜视频中学会识别息肉
Contrastive Learning under Noisy Temporal Self-Supervision for Colonoscopy Videos
- 利用结肠镜检查的时间顺序构建自监督关联，自动定义正负样本
- 在息肉检索、重识别等任务上超越已有自监督和监督方法
- 仅需27段视频训练轻量级模型，适合临床数据少的场景
学习息肉轨迹的鲁棒表征是实现多种人工智能辅助结肠镜应用的关键，包括息肉表征、自动化报告与检索。监督对比学习虽有效，但依赖准确的正负样本定义，而正确关联同一息肉的轨迹需人工标注，成本高且需专业临床知识。本文利用结肠镜检查的时序流程，从时间结构中推导自监督关联。由于时间关联可能存在噪声，我们提出一种抗噪对比损失以应对错误关联。实验表明，所学表征在多个下游任务中表现优异，包括息肉检索、重识别、大小估计和组织学分类。该方法在所有任务上均优于先前的自监督与监督基线，并达到或超过近期基础模型性能，仅使用27段视频训练轻量级编码器。代码已公开于https://github.com/lparolari/ntssl。
原文摘要 · Abstract (English)
Learning robust representations of polyp tracklets is key to enabling multiple AI-assisted colonoscopy applications, from polyp characterization to automated reporting and retrieval. Supervised contrastive learning is an effective approach for learning such representations, but it typically relies on correct positive and negative definitions. Collecting these labels requires linking tracklets that depict the same underlying polyp entity throughout the video, which is costly and demands specialized clinical expertise. In this work, we leverage the sequential workflow of colonoscopy procedures to derive self-supervised associations from temporal structure. Since temporally derived associations are not guaranteed to be correct, we introduce a noise-aware contrastive loss to account for noisy associations. We demonstrate the effectiveness of the learned representations across multiple downstream tasks, including polyp retrieval and re-identification, size estimation, and histology classification. Our method outperforms prior self-supervised and supervised baselines, and matches or exceeds recent foundation models across all tasks, using a lightweight encoder trained on only 27 videos. Code is available at https://github.com/lparolari/ntssl.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。