分析O1-preview如何解K-SAT，发现它靠调用外部求解器而非自己推理。
Fast Analysis of the OpenAI O1-Preview Model in Solving Random K-SAT Problem: Does the LLM Solve the Problem Itself or Call an External SAT Solver?
- 通过观察模型在不同问题难度下的表现，判断其是否调用外部SAT求解器。
- 模型虽使用外部求解器，却常输出错误解，表明其推理不可靠。
- 提出量化方法评估模型是真智能还是随机猜测，适合研究AI推理能力者阅读。
本文分析了OpenAI O1-preview模型在随机K-SAT实例（K∈{2,3,4}）上的表现，以问题密度α = M/N（M为子句数，N为变量数）为参数。研究发现，该模型并非直接求解，而是调用外部SAT求解器。尽管如此，模型仍会报告错误的赋值结果。本文提出并验证了一种量化方法，用于判断模型在输出布尔可满足性问题解时，是展现出微弱智能，还是仅做随机猜测。
原文摘要 · Abstract (English)
In this manuscript, I present an analysis on the performance of OpenAI O1-preview model in solving random K-SAT instances for K$\in {2,3,4}$ as a function of $α=M/N$ where $M$ is the number of clauses and $N$ is the number of variables of the satisfiable problem. I show that the model can call an external SAT solver to solve the instances, rather than solving them directly. Despite using external solvers, the model reports incorrect assignments as output. Moreover, I propose and present an analysis to quantify whether the OpenAI O1-preview model demonstrates a spark of intelligence or merely makes random guesses when outputting an assignment for a Boolean satisfiability problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。