用大模型复现人类动机性推理实验，发现其行为与人类不符。
Replicating Human Motivated Reasoning Studies with LLMs
- 复现4项人类政治动机性推理研究，测试大模型表现
- 大模型不随动机改变判断，且答非所问倾向一致
- 对观点复制和论据评估任务有重要警示意义
动机性推理——即个体在处理信息时可能更倾向于得出准确信念或期望结论——是人类认知现象的重要研究领域。然而，基础大模型是否受动机操纵影响尚不明确。我们复现了4项既有的政治动机性推理研究，发现基础大模型的行为并未表现出预期的人类特征。此外，不同模型间存在相似行为模式，例如在回答问题时倾向于回避，以及将提供论据纳入自身观点。结果表明，基础大模型可能无法模拟人类的动机性推理过程。该发现对利用大模型执行观点复现、论据评估等任务的研究者具有重要意义。
原文摘要 · Abstract (English)
Motivated reasoning - the idea that individuals processing information may be motivated to either arrive at accurate beliefs or arrive at desired conclusions - has been well-explored as a human phenomenon. However, it remains unclear whether base LLMs are affected by motivational manipulations. Replicating 4 prior political motivated reasoning studies, we find that base LLM behavior does not align with expected human behavior. Furthermore, base LLM behavior across models shares some similarities, such as when selecting to abstain from question answering and incorporating provided arguments into opinions. The results suggest that base LLMs may not emulate human motivated reasoning processes. We emphasize the importance of these findings for researchers using LLMs to for certain tasks such as opinion replication and argument assessment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。