通过对比提问从大模型中挖掘临床变量间的关联,避免直接询问的偏差。
Eliciting associations between clinical variables from LLMs via comparison questions across populations
- 用患者三元组对比问题诱导模型回答,间接获取变量关联。
- 在慢阻肺和多发性硬化数据中,关联结果稳定且具临床意义。
- 适合关注医疗决策中因果推断的研究者或临床工程师。
大型语言模型(LLM)的训练数据包含大量生物医学文献,涵盖多种患者群体。本文研究如何从中恢复患者特征之间的相关性与因果关系,作为医疗决策的关键基础。为规避直接提问的陷阱,提出基于结构化对比问题的方法,特别是患者对比三元组问题。结合对LLM表示的统计建模,无需访问模型激活值或内部结构即可估计相关性。直观上,考察当一个患者的第二变量信息被提供时,模型对第一变量相似性的判断如何变化。通过诱导提示级环境变化,获得不同亚人群的相关性估计,进而采用不变因果预测(ICP)方法识别保守的候选父节点关联。在慢性阻塞性肺病(COPD)和多发性硬化(MS)两个临床领域验证了该方法。在不同提示环境下,所提取的相关性平滑、稳定且具有临床可解释性,且在统计上显著差异,支持下游不变性检验,使ICP能够生成一组候选不变父节点关联。结果表明,通过三元组对比的间接提取可从LLM中恢复有意义的关联结构,并为从隐含相关性走向符合模型回答模式的因果陈述提供谨慎路径。
原文摘要 · Abstract (English)
The training data of large language models (LLMs) comprises a wide range of biomedical literature, reflecting data from many different patient populations. We investigate how it might be possible to recover information on correlation and causal links between patient characteristics, as a key building block for medical decision making. To avoid the pitfalls of direct elicitation, we propose an approach based on structured comparison questions, specifically patient comparison triplet questions. This is combined with a statistical model for the LLM representation that provides estimates of correlations without access to activations or model internals. Intuitively, we consider how similarity decisions of LLMs based on a first variable are affected by providing information on a second variable for one of the patients being assessed. We then induce prompt-level environment shifts to obtain correlation estimates for different subpopulations, which enables an invariant causal prediction (ICP) approach to obtain conservative candidate parent links. We demonstrate the method in two clinical domains, chronic obstructive pulmonary disease (COPD) and multiple sclerosis (MS). Across prompted environments, the elicited correlations are smooth, stable, and clinically interpretable, yet vary in a statistically significant way that supports downstream invariance testing, such that ICP provides a small set of candidate invariant parent links. These results show that indirect elicitation via triplet comparisons can recover meaningful association structure from LLMs and offer a cautious route from implicit correlations to causal statements that are congruent with LLM answering patterns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。