用专家知识生成精准提示词，提升罕见病图像零样本分类效果
Generating customized prompts for Zero-Shot Rare Event Medical Image Classification using LLM
- 基于医学领域专家知识自动生成带判别特征的定制化提示词
- 在无额外训练下实现比现有方法更优的罕见事件分类性能
- 适用于医疗数据稀缺场景，保护隐私且无需重新训练模型
由于罕见事件发生频率低，数据量不足，深度学习难以有效估计其分布。开放词汇模型通过自然语言提示实现任意类别图像分类，传统方法依赖手动设计模板（如‘一张{}的照片’）填充类别名。本文提出一种简单高效的方法，利用领域专家知识生成高度准确且上下文相关的定制提示词，包含关键判别特征。针对医学中罕见事件存在的类间差异小、类内差异大难题，该方法显著提升分类表现。所提零样本、隐私保护方案无需额外训练，在不增加数据风险的前提下超越当前最优技术。
原文摘要 · Abstract (English)
Rare events, due to their infrequent occurrences, do not have much data, and hence deep learning techniques fail in estimating the distribution for such data. Open-vocabulary models represent an innovative approach to image classification. Unlike traditional models, these models classify images into any set of categories specified with natural language prompts during inference. These prompts usually comprise manually crafted templates (e.g., 'a photo of a {}') that are filled in with the names of each category. This paper introduces a simple yet effective method for generating highly accurate and contextually descriptive prompts containing discriminative characteristics. Rare event detection, especially in medicine, is more challenging due to low inter-class and high intra-class variability. To address these, we propose a novel approach that uses domain-specific expert knowledge on rare events to generate customized and contextually relevant prompts, which are then used by large language models for image classification. Our zero-shot, privacy-preserving method enhances rare event classification without additional training, outperforming state-of-the-art techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。