发现人类和大模型在日常推理中都依赖模式匹配，而非抽象思维。
Reasoning as Pattern Matching: Shared Mechanisms in Human and LLM Everyday Reasoning
- 通过对比人类与25个大模型的推理表现，发现错误模式相似。
- 识别出模型中驱动推理的关键注意力头，其功能为模式匹配。
- 能预测人类因无关提示细节引发的意外错误，说明机制相通。
当大语言模型（LLMs）在推理中无法泛化或出现随机错误时，常被视为仅进行模式匹配而非真正推理。人们的行为则被认为不会出现此类错误，因其基于原理性、抽象的世界模型。我们评估了人类参与者和25个LLMs在多种日常情境下的常识推理能力，发现人与模型存在相似的错误模式。进一步分析发现，驱动模型响应的关键注意力头实现了一种模式匹配机制，并能预测由看似无关提示细节引发的人类推理错误。结果表明，人类和大模型的日常因果推理更符合模式匹配，而非抽象世界模型。
原文摘要 · Abstract (English)
When large language models (LLMs) fail to generalize or make haphazard errors in reasoning, it is often taken as evidence that LLMs are not truly reasoning, but rather performing a kind of pattern matching. The implication is that people's behavior does not exhibit the same types of failures because human reasoning uses principled and abstract world models. We evaluate human participants and 25 LLMs on their ability to engage in common-sense reasoning about a variety of everyday situations and observe similar patterns of errors in both people and models. We then identify the set of attention heads driving LLM responses and find that these heads implement a form of pattern-matching. These attention heads allow us to predict seemingly inexplicable reasoning errors in people caused by ostensibly irrelevant prompt details. Taken together, our results suggest that everyday causal reasoning in people and LLMs is more consistent with a form of pattern-matching than with abstract world models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。