大模型推理时易受认知偏差影响，难以自然使用有效概率思维策略。
Do Chains-of-Thoughts of Large Language Models Suffer from Hallucinations, Cognitive Biases, or Phobias in Bayesian Reasoning?
- 通过提示引导大模型使用自然频率等生态合理策略
- 模型虽能采纳但应用不一致，仍偏好符号化推理
- 适合研究模型认知偏差或改进教学型AI的读者
学习理性推理与清晰解释论证是学生发展认知、数学与计算思维的核心。这在不确定性问题和贝叶斯推理中尤为困难。新一代大语言模型（LLMs）具备链式思维（CoT）推理能力，可通过与自身内部语音对话的方式解释推理过程，为学习贝叶斯推理提供了绝佳机会。此外，不同模型有时得出相反结论，使得通过详细比较推理过程实现深度学习成为可能。然而，我们发现这些模型不会自主使用人类有效的生态策略，如自然频率、整体对象和具身启发法。这些策略有助于人类避免关键错误，并在贝叶斯推理教学中证明了其教育价值。为克服此类偏差并促进理解与学习，我们在提示中引入促使模型使用这些策略的设计。结果显示，尽管部分模型能采纳这些策略，但应用并不一致，且普遍存在对生态合理策略的回避甚至‘恐惧’，更倾向于符号化推理。
原文摘要 · Abstract (English)
Learning to reason and carefully explain arguments is central to students' cognitive, mathematical, and computational thinking development. This is particularly challenging in problems under uncertainty and in Bayesian reasoning. With the new generation of large language models (LLMs) capable of reasoning using Chain-of-Thought (CoT), there is an excellent opportunity to learn with them as they explain their reasoning through a dialogue with their artificial internal voice. It is an engaging and excellent opportunity to learn Bayesian reasoning. Furthermore, given that different LLMs sometimes arrive at opposite solutions, CoT generates opportunities for deep learning by detailed comparisons of reasonings. However, unlike humans, we found that they do not autonomously explain using ecologically valid strategies like natural frequencies, whole objects, and embodied heuristics. This is unfortunate, as these strategies help humans avoid critical mistakes and have proven pedagogical value in Bayesian reasoning. In order to overcome these biases and aid understanding and learning, we included prompts that induce LLMs to use these strategies. We found that LLMs with CoT incorporate them but not consistently. They show persistent biases towards symbolic reasoning and avoidance or phobia of ecologically valid strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。