让大模型用符号化思维高效推理,压缩16倍且更准更快。
ORION: Teaching Language Models to Reason Efficiently in the Language of Thought
- 用符号化思维(Mentalese)代替冗长文字,实现极简推理
- 在数学与规划任务上减少4-16倍token,延迟降5倍,训练成本降9倍
- 适合追求低延迟、低成本推理的落地应用
大型推理模型在数学、代码生成和任务规划中表现优异,但依赖长串冗余的“思考”文本,导致高延迟、重复和逻辑混乱。受思想语言假说启发,我们提出一个框架,让模型以类似人类符号化思维(称为Mentalese)的方式进行紧凑推理。该方法将抽象推理编码为高度压缩、结构化的符号,使模型以更少步骤解决复杂问题。为此,我们设计了短长度偏好优化(SLPO),一种强化学习方法,奖励简洁正确的解法,同时允许必要时使用更长推理。在适配Mentalese的模型上,SLPO显著提升压缩率,实现无计算开销的精准推理。在AIME 2024/2025、MinervaMath、OlympiadBench、Math500和AMC等基准测试中,ORION模型推理过程减少4-16倍token,推断延迟降低最高5倍,训练成本较DeepSeek R1 Distilled降低7-9倍,同时保持其90-98%的准确率。相比Claude和ChatGPT-4o,ORION准确率高出最多5%,压缩比达2倍。结果表明,类Mentalese的压缩推理接近人类认知效率,支持实时、低成本且不失精度的推理。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) achieve strong performance in mathematics, code generation, and task planning, but their reliance on long chains of verbose "thinking" tokens leads to high latency, redundancy, and incoherent reasoning paths. Inspired by the Language of Thought Hypothesis, which posits that human reasoning operates over a symbolic, compositional mental language called Mentalese, we introduce a framework that trains models to reason in a similarly compact style. Mentalese encodes abstract reasoning as ultra-compressed, structured tokens, enabling models to solve complex problems with far fewer steps. To improve both efficiency and accuracy, we propose SHORTER LENGTH PREFERENCE OPTIMIZATION (SLPO), a reinforcement learning method that rewards concise solutions that stay correct, while still allowing longer reasoning when needed. Applied to Mentalese-aligned models, SLPO yields significantly higher compression rates by enabling concise reasoning that preserves the benefits of detailed thinking without the computational overhead. Across benchmarks including AIME 2024 and 2025, MinervaMath, OlympiadBench, Math500, and AMC, our ORION models produce reasoning traces with 4-16x fewer tokens, achieve up to 5x lower inference latency, and reduce training costs by 7-9x relative to the DeepSeek R1 Distilled model, while maintaining 90-98% of its accuracy. ORION also surpasses Claude and ChatGPT-4o by up to 5% in accuracy while maintaining 2x compression. These results show that Mentalese-style compressed reasoning offers a step toward human-like cognitive efficiency, enabling real-time, cost-effective reasoning without sacrificing accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。