arXiv:2505.08798eess.IVcs.AI2025-05被引 3

用少量标注样本实现癌症图像分类,无需重新训练模型。

In-Context Learning for Label-Efficient Cancer Image Classification in Oncology

  • 通过上下文学习让视觉语言模型仅凭少量样本适应新诊断任务。
  • GPT-4o在二分类中达到0.81的F1分数,多分类达0.60。
  • 开源模型表现接近闭源模型,适合资源受限的临床场景。

人工智能在肿瘤学中的应用受限于对大规模标注数据的依赖及领域特定任务需重新训练模型。为此,我们探索了上下文学习(ICL)作为一种替代方案,使模型在推理时仅通过少量标注样本即可适应新诊断任务,无需参数更新。使用四种视觉语言模型(Paligemma、CLIP、ALIGN和GPT-4o),我们在三个肿瘤学数据集(MHIST、PatchCamelyon、HAM10000)上评估性能。据我们所知,这是首个在不同肿瘤分类任务中对比多种VLM性能的研究。所有模型均在少样本提示下显著提升,其中GPT-4o在二分类中达到0.81的F1分数,多分类为0.60。尽管结果低于全微调系统上限,但表明仅用少量示例即可近似特定任务行为,类比临床医生基于既往病例推理的方式。值得注意的是,如Paligemma和CLIP等开源模型虽规模较小,但仍表现出竞争力,显示其在计算资源受限的临床环境中部署的可行性。总体而言,ICL为罕见癌症及数据稀缺环境下的肿瘤诊断提供了实用解决方案。

原文摘要 · Abstract (English)

The application of AI in oncology has been limited by its reliance on large, annotated datasets and the need for retraining models for domain-specific diagnostic tasks. Taking heed of these limitations, we investigated in-context learning as a pragmatic alternative to model retraining by allowing models to adapt to new diagnostic tasks using only a few labeled examples at inference, without the need for retraining. Using four vision-language models (VLMs)-Paligemma, CLIP, ALIGN and GPT-4o, we evaluated the performance across three oncology datasets: MHIST, PatchCamelyon and HAM10000. To the best of our knowledge, this is the first study to compare the performance of multiple VLMs on different oncology classification tasks. Without any parameter updates, all models showed significant gains with few-shot prompting, with GPT-4o reaching an F1 score of 0.81 in binary classification and 0.60 in multi-class classification settings. While these results remain below the ceiling of fully fine-tuned systems, they highlight the potential of ICL to approximate task-specific behavior using only a handful of examples, reflecting how clinicians often reason from prior cases. Notably, open-source models like Paligemma and CLIP demonstrated competitive gains despite their smaller size, suggesting feasibility for deployment in computing constrained clinical environments. Overall, these findings highlight the potential of ICL as a practical solution in oncology, particularly for rare cancers and resource-limited contexts where fine-tuning is infeasible and annotated data is difficult to obtain.

上下文学习癌症图像少样本视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。