arXiv:2601.01868cs.CL2026-01被引 3

打造可信赖的皮肤科多模态模型,让AI诊断更贴近医生思维。

DermoGPT: Open Weights and Open Data for Morphology-Grounded Dermatological Reasoning MLLMs

  • 构建形态学驱动的指令数据集,覆盖从观察到诊断全流程。
  • 在3,600个专家验证的开放题上表现超越16个基线模型。
  • 开源数据与模型,适合临床研究和医疗AI开发者使用。

多模态大语言模型在医疗领域潜力巨大,但皮肤科进展受限于训练数据少、任务覆盖窄及缺乏临床真实监督。本文提出完整解决方案:首先构建包含211,243张图像和772,675条轨迹的DermoInstruct指令语料库,涵盖五种任务格式,覆盖从形态观察、临床推理到最终诊断的完整诊断流程;其次建立DermoBench基准,评估11项任务,覆盖形态、诊断、推理和公平性四维度,含3,600个专家验证的开放问答实例及人类表现基线;最后开发DermoGPT,通过监督微调结合形态一致性的强化学习目标(MAVIC)训练,并在推理时引入置信度-一致性测试时间自适应(CCT)机制。实验表明,DermoGPT在所有维度显著优于16个代表性基线,达到当前最佳性能,大幅缩小人机差距。DermoInstruct、DermoBench和DermoGPT将公开发布于https://github.com/mendicant04/DermoGPT。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) show promise for medical applications, yet progress in dermatology lags due to limited training data, narrow task coverage, and lack of clinically-grounded supervision that mirrors expert diagnostic workflows. We present a comprehensive framework to address these gaps. First, we introduce DermoInstruct, a large-scale morphology-anchored instruction corpus comprising 211,243 images and 772,675 trajectories across five task formats, capturing the complete diagnostic pipeline from morphological observation and clinical reasoning to final diagnosis. Second, we establish DermoBench, a rigorous benchmark evaluating 11 tasks across four clinical axes: Morphology, Diagnosis, Reasoning, and Fairness, including a challenging subset of 3,600 expert-verified open-ended instances and human performance baselines. Third, we develop DermoGPT, a dermatology reasoning MLLM trained via supervised fine-tuning followed by our Morphologically-Anchored Visual-Inference-Consistent (MAVIC) reinforcement learning objective, which enforces consistency between visual observations and diagnostic conclusions. At inference, we deploy Confidence-Consistency Test-time adaptation (CCT) for robust predictions. Experiments show DermoGPT significantly outperforms 16 representative baselines across all axes, achieving state-of-the-art performance while substantially narrowing the human-AI gap. DermoInstruct, DermoBench and DermoGPT will be made publicly available at https://github.com/mendicant04/DermoGPT upon acceptance.

皮肤科AI多模态模型医学推理开源数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。