arXiv:2601.05376cs.AIcs.CL2026-01被引 4

医学人格设定能提升重症场景表现,却可能损害全科诊疗效果。

The Persona Paradox: Medical Personas as Behavioral Priors in Clinical Language Models

  • 用医生/护士等角色和胆大/谨慎风格控制临床模型行为
  • 重症任务准确率最高提升20%,全科任务表现反而下降近同等幅度
  • 专家对安全性的判断分歧大,模型风险倾向高度依赖具体模型

将角色设定视为大语言模型的行为先验,常被认为能单调提升专业性和安全性。然而其在高风险临床决策中的影响尚不明确。我们系统评估了临床大模型中基于角色的控制机制,考察不同职业身份(如急诊医生、护士)与交互风格(大胆或谨慎)如何影响模型在多种医疗任务中的表现。通过多维度评估任务准确性、校准度及安全相关风险行为,发现其效果具有显著情境依赖性且非单调:在危重症任务中,医学角色设定可使准确率和校准度提升约20%,但在初级医疗场景中表现反而下降相当幅度。交互风格调节风险倾向和敏感性,但效果高度依赖模型本身。尽管聚合的模型评判偏好医学角色,但人类临床专家对安全合规性仅达中等一致(平均Cohen's κ=0.43),且95.9%的判断对推理质量信心不足。结果表明,角色设定作为行为先验引入的是情境依赖的权衡,而非安全或专业的保证。代码已开源。

原文摘要 · Abstract (English)

Persona conditioning can be viewed as a behavioral prior for large language models (LLMs) and is often assumed to confer expertise and improve safety in a monotonic manner. However, its effects on high-stakes clinical decision-making remain poorly characterized. We systematically evaluate persona-based control in clinical LLMs, examining how professional roles (e.g., Emergency Department physician, nurse) and interaction styles (bold vs.\ cautious) influence behavior across models and medical tasks. We assess performance on clinical triage and patient-safety tasks using multidimensional evaluations that capture task accuracy, calibration, and safety-relevant risk behavior. We find systematic, context-dependent, and non-monotonic effects: Medical personas improve performance in critical care tasks, yielding gains of up to $\sim+20\%$ in accuracy and calibration, but degrade performance in primary-care settings by comparable margins. Interaction style modulates risk propensity and sensitivity, but it's highly model-dependent. While aggregated LLM-judge rankings favor medical over non-medical personas in safety-critical cases, we found that human clinicians show moderate agreement on safety compliance (average Cohen's $κ= 0.43$) but indicate a low confidence in 95.9\% of their responses on reasoning quality. Our work shows that personas function as behavioral priors that introduce context-dependent trade-offs rather than guarantees of safety or expertise. The code is available at https://github.com/rsinghlab/Persona\_Paradox.

临床LLM角色设定行为先验医学AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。