用置信度指导自蒸馏,小模型也能精准分割肠息肉
The Power of Certainty: How Confident Models Lead to Better Segmentation
- 基于迭代置信度动态调整损失,仅用历史数据无需额外计算
- 在多中心数据集上超越现有模型,平均Dice达90.1%
- 适合医疗影像领域,尤其对参数受限的部署场景友好
深度学习模型已被用于结肠镜检查中自动检测和精确分割肠息肉。尽管这些先进模型性能优异,但通常需要大量参数,易因训练数据偏差导致过拟合,且跨数据集泛化能力差。知识蒸馏与自蒸馏被视为缓解大模型局限性的有效策略,但往往资源消耗大,需多个模型及大量内存。本文提出一种基于置信度的自蒸馏方法,仅利用训练过程中的前一迭代数据存储,无需额外计算或内存开销。通过动态置信系数计算批次内前后迭代间的损失,实现高效优化。在肠息肉分割任务上进行综合实验验证,所提方法超越当前最优模型,并在多个临床中心采集的数据集上表现出良好泛化性。代码将在论文被接受后公开。
原文摘要 · Abstract (English)
Deep learning models have been proposed for automatic polyp detection and precise segmentation of polyps during colonoscopy procedures. Although these state-of-the-art models achieve high performance, they often require a large number of parameters. Their complexity can make them prone to overfitting, particularly when trained on biased datasets, and can result in poor generalization across diverse datasets. Knowledge distillation and self-distillation are proposed as promising strategies to mitigate the limitations of large, over-parameterized models. These approaches, however, are resource-intensive, often requiring multiple models and significant memory during training. We propose a confidence-based self-distillation approach that outperforms state-of-the-art models by utilizing only previous iteration data storage during training, without requiring extra computation or memory usage during testing. Our approach calculates the loss between the previous and current iterations within a batch using a dynamic confidence coefficient. To evaluate the effectiveness of our approach, we conduct comprehensive experiments on the task of polyp segmentation. Our approach outperforms state-of-the-art models and generalizes well across datasets collected from multiple clinical centers. The code will be released to the public once the paper is accepted.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。