arXiv:2505.03350cs.CV2025-05

用图文模型提升肝脏病灶分类,小样本下表现更优

A Vision-Language Model for Focal Liver Lesion Classification

  • 将病灶类别信息融入文本编码器,不增加推理开销
  • 在MPCT-FLLs数据集上准确率和AUC均优于CLIP与MedCLIP
  • 轻量ResNet18骨干网络在数据少时仍具强分类能力

准确分类局灶性肝病变对肝病诊疗至关重要。传统监督深度学习依赖大规模标注数据,而医学影像中此类数据常受限。近期视觉语言模型(如CLIP)被用于图像分类,相比仅依赖视觉信息的卷积神经网络,其通过图文多模态学习,在少量标注数据下仍能有效建模。受CLIP启发,我们提出专用于局灶性肝病变(FLLs)分类的Liver-VLM模型:首先,将类别信息嵌入文本编码器,不增加推理开销;其次,通过计算图像与文本嵌入间的成对余弦相似度,并以交叉熵损失优化,实现图像特征与类别级文本特征的有效对齐。在MPCT-FLLs数据集上的实验表明,Liver-VLM在准确率和曲线下面积(AUC)上均优于标准CLIP与MedCLIP模型。进一步分析显示,采用轻量级ResNet18骨干网络可显著提升分类性能,尤其在数据受限条件下表现更佳。

原文摘要 · Abstract (English)

Accurate classification of focal liver lesions is crucial for diagnosis and treatment in hepatology. However, traditional supervised deep learning models depend on large-scale annotated datasets, which are often limited in medical imaging. Recently, Vision-Language models (VLMs) such as Contrastive Language-Image Pre-training model (CLIP) has been applied to image classifications. Compared to the conventional convolutional neural network (CNN), which classifiers image based on visual information only, VLM leverages multimodal learning with text and images, allowing it to learn effectively even with a limited amount of labeled data. Inspired by CLIP, we pro-pose a Liver-VLM, a model specifically designed for focal liver lesions (FLLs) classification. First, Liver-VLM incorporates class information into the text encoder without introducing additional inference overhead. Second, by calculating the pairwise cosine similarities between image and text embeddings and optimizing the model with a cross-entropy loss, Liver-VLM ef-fectively aligns image features with class-level text features. Experimental results on MPCT-FLLs dataset demonstrate that the Liver-VLM model out-performs both the standard CLIP and MedCLIP models in terms of accuracy and Area Under the Curve (AUC). Further analysis shows that using a lightweight ResNet18 backbone enhances classification performance, particularly under data-constrained conditions.

肝病诊断图文模型小样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。