arXiv:2412.10509cs.AIcs.CL2024-12被引 1

大模型在因果推理中会陷入虚假因果偏见,尤其在否定因果条件时更明显。

Do Large Language Models Show Biases in Causal Learning?

  • 构建2000+样本数据集,测试模型在相关性、无关联和时间错位场景下的因果判断。
  • 在时间错位或零关联场景下,模型对因果关系的误判率显著高于人类平均水平。
  • 发现大模型未真正掌握因果推理的规范原则,易受虚假关联误导。

因果学习是基于已有信息进行因果推断的认知过程,常受认知偏差影响,如虚假因果幻觉——在缺乏证据时仍感知变量间存在因果关系。该偏差被认为与社会偏见、刻板印象、虚假信息传播及迷信思维有关。本文研究大语言模型(LLMs)在真实世界与受控实验情境中的因果推理是否存在此类偏差。我们构建了包含超过2000个样本的数据集,涵盖纯相关性、零关联性和时间顺序不符(效应早于原因)等情形。通过提示模型生成回答或评分(0-100),评估其在结构化场景下的错误因果推断倾向。结果表明,大模型存在强烈的因果幻觉偏差:在开放生成任务中,面对虚假相关性时,其偏差水平与人类相当甚至更低;但在零关联或时间顺序矛盾的情境下,要求量化判断时,模型偏差显著升高。这说明大模型尚未一致、可靠地内化准确因果学习所需的规范原则。

原文摘要 · Abstract (English)

Causal learning is the cognitive process of developing the capability of making causal inferences based on available information, often guided by normative principles. This process is prone to errors and biases, such as the illusion of causality, in which people perceive a causal relationship between two variables despite lacking supporting evidence. This cognitive bias has been proposed to underlie many societal problems, including social prejudice, stereotype formation, misinformation, and superstitious thinking. In this research, we investigate whether large language models (LLMs) develop causal illusions, both in real-world and controlled laboratory contexts of causal learning and inference. To this end, we built a dataset of over 2K samples including purely correlational cases, situations with null contingency, and cases where temporal information excludes the possibility of causality by placing the potential effect before the cause. We then prompted the models to make statements or answer causal questions to evaluate their tendencies to infer causation erroneously in these structured settings. Our findings show a strong presence of causal illusion bias in LLMs. Specifically, in open-ended generation tasks involving spurious correlations, the models displayed bias at levels comparable to, or even lower than, those observed in similar studies on human subjects. However, when faced with null-contingency scenarios or temporal cues that negate causal relationships, where it was required to respond on a 0-100 scale, the models exhibited significantly higher bias. These findings suggest that the models have not uniformly, consistently, or reliably internalized the normative principles essential for accurate causal learning.

大模型因果推理偏见检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。