发现大模型在急诊分诊中受隐含变量影响产生歧视性判断
Uncovering Latent Bias in LLM-Based Emergency Department Triage Through Proxy Variables
- 用32个患者代理变量测试模型对不同背景患者的判断偏差
- 发现模型会因输入中特定词汇改变对病情严重性的评估
- 适合关注医疗AI公平性与安全性的研究者和临床工程师
大型语言模型（LLM）在临床决策中的应用日益广泛，但针对不同种族、社会经济及临床背景患者的隐性偏见依然存在。本研究探讨了基于LLM的急诊分诊系统中的偏见问题，采用32个患者层面的代理变量，每个变量以正负双重表述形式呈现，并在公开数据集（MIMIC-IV-ED Demo、MIMIC-IV Demo）和受限访问凭证数据集（MIMIC-IV-ED 和 MIMIC-IV）上进行评估。结果表明，在急诊分诊场景中，模型行为受代理变量影响而表现出歧视性倾向；且当输入上下文中出现特定词元时，无论其正负语义，模型均系统性地改变对患者病情严重程度的判断。这说明当前模型仍依赖于噪声大、非因果的信号，无法准确反映真实病情严重程度。因此，亟需改进以确保人工智能技术在临床环境中的安全与负责任部署。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have enabled their integration into clinical decision-making; however, hidden biases against patients across racial, social, economic, and clinical backgrounds persist. In this study, we investigate bias in LLM-based medical AI systems applied to emergency department (ED) triage. We employ 32 patient-level proxy variables, each represented by paired positive and negative qualifiers, and evaluate their effects using both public (MIMIC-IV-ED Demo, MIMIC-IV Demo) and restricted-access credentialed (MIMIC-IV-ED and MIMIC-IV) datasets as appropriate~\cite{mimiciv_ed_demo,mimiciv_ed,mimiciv}. Our results reveal discriminatory behavior mediated through proxy variables in ED triage scenarios, as well as a systematic tendency for LLMs to modify perceived patient severity when specific tokens appear in the input context, regardless of whether they are framed positively or negatively. These findings indicate that AI systems is still imperfectly trained on noisy, sometimes non-causal signals that do not reliably reflect true patient acuity. Consequently, more needs to be done to ensure the safe and responsible deployment of AI technologies in clinical settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。