PulseMind整合多模态数据与动态对话,提升真实临床诊断的准确性。
PulseMind: A Multi-Modal Medical Model for Real-World Clinical Diagnosis
- 构建多轮诊疗数据集MediScope,含9.8万次真实会诊与60万张医学图像。
- 提出四维评估框架,以主动性、准确性等指标验证模型在真实场景中的表现。
- 采用相对偏好优化策略,使训练更稳定且贴合医生实际判断逻辑。
当前医疗多模态模型主要聚焦于皮肤科、病理科或放射科等特定图像分析,难以全面反映真实临床诊断中多元输入与持续上下文理解的需求。为此,我们提出PulseMind,一套全新的多模态诊断模型体系,包含系统化构建的数据集、全面的评估基准与定制化训练框架。首先,我们构建了MediScope数据集,涵盖98,000次真实多轮诊疗会话与601,500张医学图像,覆盖10个主要临床科室及200多个亚专科。其次,为更真实反映临床需求,我们设计了PulseMind Benchmark,一个支持多轮诊断咨询的基准测试,采用主动性、准确性、实用性与语言质量四个维度的评估协议。最后,我们提出基于对比强化政策优化(CRPO)的训练框架,通过多维度比较产生的相对偏好信号,替代绝对评分奖励,提供更稳定且符合人类判断的训练指导。大量实验表明,PulseMind在诊断咨询基准与公开医疗基准上均达到竞争性性能。
原文摘要 · Abstract (English)
Recent advances in medical multi-modal models focus on specialized image analysis like dermatology, pathology, or radiology. However, they do not fully capture the complexity of real-world clinical diagnostics, which involve heterogeneous inputs and require ongoing contextual understanding during patient-physician interactions. To bridge this gap, we introduce PulseMind, a new family of multi-modal diagnostic models that integrates a systematically curated dataset, a comprehensive evaluation benchmark, and a tailored training framework. Specifically, we first construct a diagnostic dataset, MediScope, which comprises 98,000 real-world multi-turn consultations and 601,500 medical images, spanning over 10 major clinical departments and more than 200 sub-specialties. Then, to better reflect the requirements of real-world clinical diagnosis, we develop the PulseMind Benchmark, a multi-turn diagnostic consultation benchmark with a four-dimensional evaluation protocol comprising proactiveness, accuracy, usefulness, and language quality. Finally, we design a training framework tailored for multi-modal clinical diagnostics, centered around a core component named Comparison-based Reinforcement Policy Optimization (CRPO). Compared to absolute score rewards, CRPO uses relative preference signals from multi-dimensional com-parisons to provide stable and human-aligned training guidance. Extensive experiments demonstrate that PulseMind achieves competitive performance on both the diagnostic consultation benchmark and public medical benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。