用文字描述解剖知识提升器官分割准确率
Leveraging Textual Anatomical Knowledge for Class-Imbalanced Semi-Supervised Multi-Organ Segmentation
- 将GPT-4生成的解剖描述转为文本特征注入分割头
- 在多个数据集上实现更高分割精度，显著优于现有方法
- 适合医学图像分割、标注稀缺场景下使用
3D医学图像标注耗时且需专业知识，推动了半监督学习（SSL）在分割任务中的应用。然而，器官复杂的解剖结构常导致严重类别不平衡，制约了SSL在真实场景中的部署。尽管存在器官相对位置、形状先验等有价值先验信息，现有SSL方法尚未充分挖掘。为此，我们提出一种新方法，将文本解剖知识（TAK）融入分割模型：利用GPT-4o生成解剖先验的文本描述，并通过CLIP模型编码，作为分割头参数注入；同时采用对比学习增强文本先验与视觉特征的对齐。大量实验表明，该方法性能显著优于现有最先进方法。代码将开源于：https://github.com/Lunn88/TAK-Semi。
原文摘要 · Abstract (English)
Annotating 3D medical images demands substantial time and expertise, driving the adoption of semi-supervised learning (SSL) for segmentation tasks. However, the complex anatomical structures of organs often lead to significant class imbalances, posing major challenges for deploying SSL in real-world scenarios. Despite the availability of valuable prior information, such as inter-organ relative positions and organ shape priors, existing SSL methods have yet to fully leverage these insights. To address this gap, we propose a novel approach that integrates textual anatomical knowledge (TAK) into the segmentation model. Specifically, we use GPT-4o to generate textual descriptions of anatomical priors, which are then encoded using a CLIP-based model. These encoded priors are injected into the segmentation model as parameters of the segmentation head. Additionally, contrastive learning is employed to enhance the alignment between textual priors and visual features. Extensive experiments demonstrate the superior performance of our method, significantly surpassing state-of-the-art approaches. The source code will be available at: https://github.com/Lunn88/TAK-Semi.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。