用生成图像缓解皮肤癌诊断中的种族偏见
Prompting Medical Vision-Language Models to Mitigate Diagnosis Bias by Generating Realistic Dermoscopic Images
- 用视觉语言模型生成精准提示，驱动扩散模型合成真实皮肤镜图像
- 在不平衡数据集上提升暗肤色人群的诊断表现
- 适合关注医疗AI公平性的研究者与临床开发者
皮肤疾病诊断中的人工智能已显著进步，但对肤色等敏感属性存在明显偏差。为此，我们提出新型生成式框架Dermatology Diffusion Transformer（DermDiT），利用视觉语言模型生成每张皮肤镜图像的准确文本提示，结合多模态图文学习生成合成图像。该方法增强数据集中代表性不足群体（如特定肤色患者或疾病类型）的表征，改善高度不平衡数据下的临床诊断性能。大量实验表明，大视觉语言模型提供的深层语义信息显著提升了DermDiT生成图像的质量。代码已开源：https://github.com/Munia03/DermDiT。
原文摘要 · Abstract (English)
Artificial Intelligence (AI) in skin disease diagnosis has improved significantly, but a major concern is that these models frequently show biased performance across subgroups, especially regarding sensitive attributes such as skin color. To address these issues, we propose a novel generative AI-based framework, namely, Dermatology Diffusion Transformer (DermDiT), which leverages text prompts generated via Vision Language Models and multimodal text-image learning to generate new dermoscopic images. We utilize large vision language models to generate accurate and proper prompts for each dermoscopic image which helps to generate synthetic images to improve the representation of underrepresented groups (patient, disease, etc.) in highly imbalanced datasets for clinical diagnoses. Our extensive experimentation showcases the large vision language models providing much more insightful representations, that enable DermDiT to generate high-quality images. Our code is available at https://github.com/Munia03/DermDiT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。