用三段论测试大模型逻辑演进，发现规模和思考机制促其向现代逻辑转变
A Syllogistic Probe: Tracing the Evolution of Logic Reasoning in Large Language Models
- 以存在蕴涵为探针，对比传统与现代逻辑下的三段论推理
- 模型规模越大，越倾向现代逻辑；思维链可加速这一转变
- 基础模型架构决定逻辑演进的难易与稳定性，适合研究模型认知机制
人类逻辑从直觉推理逐步转向严谨的形式系统。受大语言模型（LLMs）近期进展启发，我们探究了LLMs是否也展现出类似的内在逻辑框架演化。通过存在蕴涵作为探针，评估在传统逻辑与现代逻辑下的三段论推理表现。在新构建的三段论数据集上对当前主流大模型进行广泛实验，发现：(i) 模型规模扩大促进向现代逻辑的转变；(ii) 思维链（thinking）能有效加速这一过程，超越参数量扩展的效应；(iii) 基础模型架构在决定该转变是否容易、稳定出现方面起关键作用。此外，还开展了额外实验，深入分析当前大模型在三段论推理中的各项特性。
原文摘要 · Abstract (English)
Human logic has gradually shifted from intuition-driven inference to rigorous formal systems. Motivated by recent advances in large language models (LLMs), we explore whether LLMs exhibit a similar evolution in the underlying logical framework. Using existential import as a probe, we for evaluate syllogism under traditional and modern logic. Through extensive experiments of testing SOTA LLMs on a new syllogism dataset, we have some interesting findings: (i) Model size scaling promotes the shift toward modern logic; (ii) Thinking serves as an efficient accelerator beyond parameter scaling; (iii) the Base model plays a crucial role in determining how easily and stably this shift can emerge. Beyond these core factors, we conduct additional experiments for in-depth analysis of properties of current LLMs on syllogistic reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。