用真实对话策略训练患者模拟器,揭示问诊与诊断的相互制约关系。
Exploring the Inquiry-Diagnosis Relationship with Advanced Patient Simulators
- 基于真实医患对话提取动态问诊策略,构建更逼真的患者模拟器。
- 实验证明问诊不足会严重限制诊断效果,符合利比格定律。
- 区分四种问诊类型,解释不同模型表现差异,适合医疗AI评估研究者。
大语言模型在在线医疗咨询中展现出巨大潜力,但多数研究聚焦于信息充分条件下的诊断准确率,忽视了问诊环节。现有方法多依赖提示工程构建患者代理,难以真实模拟患者行为。本文从真实医患对话中提取对话策略,用于训练患者模拟器,该模拟器具备更高拟人度和更低幻觉率。这一创新使诊断模型评估更准确,并生成真实合成数据。我们在多个场景下开展实验,发现问诊与诊断遵循利比格定律:无论诊断能力如何,问诊不足都会限制诊断效果;反之亦然。实验还揭示模型间问诊表现存在显著差异。为此,我们将问诊过程分为四类,分析其分布有助于解释性能差异。模型权重已开源:https://github.com/PatientSimulator/PatientSimulator。
原文摘要 · Abstract (English)
Recently, large language models have shown great potential to transform online medical consultation. Despite this, most research targets improving diagnostic accuracy with ample information, often overlooking the inquiry phase. Some studies try to evaluate or refine doctor models by using prompt-engineered patient agents. However, prompt engineering alone falls short in accurately simulating real patients. We need to explore new paradigms for patient simulation. Furthermore, the relationship between inquiry and diagnosis remains unexplored. This paper extracts dialogue strategies from real doctor-patient conversations to guide the training of a patient simulator. Our simulator shows higher anthropomorphism and lower hallucination rates, using dynamic dialogue strategies. This innovation offers a more accurate evaluation of diagnostic models and generates realistic synthetic data. We conduct extensive experiments on the relationship between inquiry and diagnosis, showing they adhere to Liebig's law: poor inquiry limits diagnosis effectiveness, regardless of diagnostic skill, and vice versa. The experiments also reveal substantial differences in inquiry performance among models. To delve into this phenomenon, the inquiry process is categorized into four distinct types. Analyzing the distribution of inquiries across these types helps explain the performance differences. The weights of our patient simulator are available https://github.com/PatientSimulator/PatientSimulator.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。