用符号推理层提升大模型长程任务能力,无需训练即可大幅提效
CoreThink: A Symbolic Reasoning Layer to reason over Long Horizon Tasks with LLMs
- 基于通用符号推理新方法构建推理层,聚焦工具调用、代码生成与规划
- 在多个基准上达成最新纪录,如Livecodebench达66.66%,SWE-Bench Lite达62.3%
- 无需微调或训练成本,纯性能提升,适合高要求推理场景
我们提出CoreThink,一种基于新型推理方法‘通用符号’(General Symbolics)的先进推理层。该方法区别于测试时扩展、监督微调和可验证奖励强化学习等范式。核心符号推理器(GSR)专门针对工具调用、代码生成和规划三大应用场景,在七个相关基准上表现优异。尤其在Livecodebench v6上达到66.66%的SOTA得分,在Instruction-Following Evals上达89%,在ARC-AGI-2上达24.4%。我们还基于通用符号原理开发了一款智能编码集成开发环境,其在SWE-Bench Lite上实现62.3%的SOTA准确率。所有改进均无需微调或训练成本。推理层设计确保模型在推理任务上的准确性不会下降,且能持续提供纯性能提升。我们主张现有方法终将陷入收益递减,亟需发展新型推理技术。本技术报告概述了整体思路,并开放CoreThink模型供推理密集型应用使用。
原文摘要 · Abstract (English)
We introduce CoreThink, a state-of-the-art Reasoning Layer built upon a novel reasoning method called General Symbolics. This approach diverges from reasoning paradigms such as test-time scaling, Supervised Fine-Tuning (SFT), and Reinforcement Learning with Verifiable Rewards (RLVR). CoreThink General Symbolic Reasoner (GSR) is specifically structured around three key use cases: tool-calling, code generation, and planning, demonstrating exemplary performance across a total of seven benchmarks in their respective areas. Notably, we are achieving SOTA scores of 66.66% on Livecodebench v6, 89% on Instruction-Following Evals, and 24.4% on ARC-AGI-2. We also present an agentic coding IDE, developed using the principles of General Symbolics, which achieves a state-of-the-art accuracy of 62.3% on SWE-Bench Lite. We are able to achieve these improvements without any fine-tuning or training costs. Our Reasoning Layer is designed to provide a pure performance uplift, ensuring that a model's accuracy on reasoning tasks is never negatively impacted. We argue that incumbent methods will eventually lead to diminishing returns in LLM performance, necessitating the development of new reasoning techniques. This technical report details our approach at a high level and the availability of the CoreThink models for reasoning-intensive use cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。