医疗大模型对提示词微小变化敏感,可能引发误诊或错误用药。
When Large Language Models Fail in Healthcare: Evaluating Sensitivity to Prompt Variations

- 通过自然与对抗性提示扰动测试模型鲁棒性
- 微小改写导致诊断结果变化,对抗攻击可诱导危险输出
- 通用与医学专用模型均存在安全风险,临床应用需谨慎
大型语言模型(LLMs)在临床问答、辅助诊断和报告摘要等医疗任务中应用日益广泛。然而,这些模型对提示词的细微变化(词汇与句法层面)高度敏感,危及临床安全。本研究基于MedMCQA基准,系统评估通用型(如GPT-3.5、Llama3)与医学专用型(如ClinicalBERT、BioLlama3、BioBERT)模型的鲁棒性。我们将扰动分为自然与对抗两类,考察其对模型一致性、准确性和可靠性的影响。结果表明,医学类LLMs并非内在安全:即使微小措辞变化也可能改变临床建议,针对性对抗提示可引发有害输出。在高风险医疗场景中,因输入重述导致诊断变更或虚构药物的行为不可接受。尽管模型对简单同义替换或改写具有一定韧性,但在句法重组或误导性语境下常失效。这种脆弱性在通用与领域专用模型中普遍存在。值得注意的是,对抗操纵可能导致推荐错误剂量或遗漏关键发现等临床危险后果。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly used in healthcare for tasks such as clinical question answering, diagnosis support, and report summarization. Despite their promise, these models remain highly sensitive to subtle prompt perturbations, both lexical and syntactic, posing serious risks in safety-critical clinical applications. In this study, we conduct a systematic sensitivity analysis to evaluate the robustness of both general-purpose (e.g., GPT-3.5, Llama3) and medical-specific LLMs (e.g., ClinicalBERT, BioLlama3, BioBERT) using the MedMCQA benchmark. We categorize perturbations into natural and adversarial types and examine their effect on model consistency, accuracy, and reliability in clinical reasoning tasks. Our findings reveal that medical LLMs are not intrinsically safe. Even minor variations in phrasing can alter clinical advice, and targeted adversarial prompts can provoke harmful outputs. In high-stakes settings like healthcare, such unpredictability is unacceptable-models that change diagnoses due to reworded inputs or hallucinate medications when slightly rephrased cannot be reliably trusted by clinicians. While models tend to show resilience to simple lexical substitutions or paraphrasing, they often break down under syntactic reordering or misleading contextual cues. This fragility is evident across both general-purpose and domain-specific LLMs. Notably, adversarial manipulations can lead to clinically dangerous outputs, such as recommending incorrect dosages or omitting critical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。