PENCIL让小模型用短记忆解决复杂问题，自动清理无用推理步骤。
PENCIL: Long Thoughts with Short Memory
- 通过递归清理机制，在生成中主动删除冗余思考
- 2500万参数模型用2048上下文解出爱因斯坦谜题
- 适合资源受限场景下的复杂推理任务
尽管当前大语言模型在使用长链式思维（CoT）提升推理能力方面表现优异，但在测试时扩展至更复杂问题仍受制于低效的内存使用——中间计算结果会持续累积在上下文中，即使已不再需要。我们提出PENCIL，将一种新颖的简化机制融入自回归生成过程，通过训练学习到的模式递归清理中间思考内容。通过迭代生成与清除思考，PENCIL能在更短上下文和更低算力下进行深度推理。实验表明，PENCIL显著优于传统CoT；例如，仅使用2500万参数的Transformer和2048上下文长度，即可解决爱因斯坦谜题——这一任务对更大模型如GPT-4也具挑战性。理论上，我们证明PENCIL可模拟任意图灵机，以最优的时间和空间复杂度实现通用高效计算，从而解决原本对标准CoT不可行的任意可计算任务。
原文摘要 · Abstract (English)
While state-of-the-art LLMs have demonstrated great promise of using long Chains-of-Thought (CoT) to boost reasoning, scaling it up to more challenging problems at test-time is fundamentally limited by suboptimal memory usage -- intermediate computations accumulate indefinitely in context even when no longer needed for future thoughts. We introduce PENCIL, which incorporates a novel reduction mechanism into the autoregressive generation process that recursively cleans up intermediate thoughts based on patterns learned from training. By iteratively generating and erasing thoughts, PENCIL can think deeper to solve harder problems using shorter context and less compute. Empirically, we observe PENCIL is significantly more effective and efficient than CoT. For example, we demonstrate PENCIL with a small 25M-parameter transformer and 2048 context length solves Einstein's puzzle -- a task that challenges much larger models like GPT-4. Theoretically, we prove PENCIL can perform universal efficient computation by simulating any Turing machines with optimal time and space complexity, and thus can solve arbitrary computable tasks that are otherwise intractable for vanilla CoT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。