通过解耦风格与内容信息,提升内窥镜图像分割的跨模态泛化能力。
Domain Generalization for Endoscopic Image Segmentation by Disentangling Style-Content Information and SuperPixel Consistency
- 用实例归一化与选择性白化解耦图像风格与内容特征
- 在息肉和巴雷特食管数据集上性能超越基线14%~18%
- 适合需要跨成像模态部署的医疗图像分割场景
频繁监测对评估胃肠道癌前病变风险至关重要。临床中常使用白光成像(WLI)、窄带成像(NBI)和荧光成像等多模态方式评估风险区域。然而,传统深度学习模型在跨模态测试时因域差距导致性能下降。此前我们提出基于超像素的SUPRA方法,利用颜色与空间距离学习域不变特征,但其聚合未充分利用结构信息,对息肉及颜色异质区域分割效果不佳。为此,本文结合实例归一化与实例选择性白化(ISW),实现风格-内容解耦,增强与SUPRA协同的域泛化能力。我们在两个数据集——EndoUDA Barrett's Esophagus 和 EndoUDA polyps 上评估,对比三种SOTA方法。结果表明,本方法在目标域上显著优于基线与现有先进方法:在息肉数据集上分别提升14%、10%、8%和18%;在巴雷特食管数据集上超越第二佳方法(EndoUDA)近2%。
原文摘要 · Abstract (English)
Frequent monitoring is necessary to stratify individuals based on their likelihood of developing gastrointestinal (GI) cancer precursors. In clinical practice, white-light imaging (WLI) and complementary modalities such as narrow-band imaging (NBI) and fluorescence imaging are used to assess risk areas. However, conventional deep learning (DL) models show degraded performance due to the domain gap when a model is trained on one modality and tested on a different one. In our earlier approach, we used a superpixel-based method referred to as "SUPRA" to effectively learn domain-invariant information using color and space distances to generate groups of pixels. One of the main limitations of this earlier work is that the aggregation does not exploit structural information, making it suboptimal for segmentation tasks, especially for polyps and heterogeneous color distributions. Therefore, in this work, we propose an approach for style-content disentanglement using instance normalization and instance selective whitening (ISW) for improved domain generalization when combined with SUPRA. We evaluate our approach on two datasets: EndoUDA Barrett's Esophagus and EndoUDA polyps, and compare its performance with three state-of-the-art (SOTA) methods. Our findings demonstrate a notable enhancement in performance compared to both baseline and SOTA methods across the target domain data. Specifically, our approach exhibited improvements of 14%, 10%, 8%, and 18% over the baseline and three SOTA methods on the polyp dataset. Additionally, it surpassed the second-best method (EndoUDA) on the Barrett's Esophagus dataset by nearly 2%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。