用文本生成皮肤病变图像,提升分类准确率最多9%。
DermaFlux: Synthetic Skin Lesion Generation with Rectified Flows for Enhanced Image Classification
- 基于修正流模型,根据描述生成真实感皮肤病变图。
- 小样本训练下分类准确率提升6%,优于扩散模型生成数据。
- 适合医学图像数据少的场景,尤其对皮肤病分类研究者有用。
尽管深度生成模型取得进展,皮肤病变分类系统仍受限于大型、多样且标注良好的临床数据集稀缺,导致良性与恶性病变类别不平衡,进而影响泛化性能。我们提出DermaFlux,一种基于修正流的文本到图像生成框架,可从皮肤病学特征的自然语言描述中合成具有临床意义的皮肤病变图像。基于Flux.1构建,使用参数高效低秩适应(LoRA)在大量公开临床图像数据集上进行微调。通过Llama 3.2生成符合皮肤病学标准(如不对称性、边界不规则、颜色变异)的合成文本描述,构建图像-文本配对。实验表明,DermaFlux生成的图像具有多样性与临床相关性;当用于增强小型真实数据集时,二分类性能提升最高达6%;若以生成图像训练分类器而非扩散模型生成图像,则提升最高达9%。仅用2,500张真实图像和4,375张DermaFlux生成样本微调ImageNet预训练ViT,即可达到78.04%的二分类准确率和0.859的AUC,超越现有最佳皮肤病模型8%。
原文摘要 · Abstract (English)
Despite recent advances in deep generative modeling, skin lesion classification systems remain constrained by the limited availability of large, diverse, and well-annotated clinical datasets, resulting in class imbalance between benign and malignant lesions and consequently reduced generalization performance. We introduce DermaFlux, a rectified flow-based text-to-image generative framework that synthesizes clinically grounded skin lesion images from natural language descriptions of dermatological attributes. Built upon Flux.1, DermaFlux is fine-tuned using parameter-efficient Low-Rank Adaptation (LoRA) on a large curated collection of publicly available clinical image datasets. We construct image-text pairs using synthetic textual captions generated by Llama 3.2, following established dermatological criteria including lesion asymmetry, border irregularity, and color variation. Extensive experiments demonstrate that DermaFlux generates diverse and clinically meaningful dermatology images that improve binary classification performance by up to 6% when augmenting small real-world datasets, and by up to 9% when classifiers are trained on DermaFlux-generated synthetic images rather than diffusion-based synthetic images. Our ImageNet-pretrained ViT fine-tuned with only 2,500 real images and 4,375 DermaFlux-generated samples achieves 78.04% binary classification accuracy and an AUC of 0.859, surpassing the next best dermatology model by 8%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。