让大模型自己写推理程序，高效解决复杂任务
Self-Steering Language Models
- 用规划器生成任务专属推理代码，由多个跟随者并行执行
- 小模型（17亿）在难题上超越大模型（GPT-4o、o1）
- 无需微调，自动实现高效搜索，适合高并发推理场景
尽管测试时推理使语言模型能够处理复杂任务，但在自然语言中进行搜索或规划仍可能缓慢、昂贵且易出错。然而，即使语言模型难以精确模拟解决问题所需的推理步骤，它们通常仍擅长描述问题的抽象结构——包括如何验证解以及如何搜索解。本文提出DisCIPL方法，实现语言模型的“自引导”：由一个规划器模型生成特定任务的推理程序，由一组跟随者模型执行。该方法赋予语言模型编写递归搜索程序以引导自身推理的能力，从而实现可验证且高效的推理。当使用小型跟随者模型（如Llama-3.2-1B或Qwen3-1.7B）时，DisCIPL在具有挑战性的约束生成任务上达到甚至超过大型模型（如GPT-4o和o1）的性能。本工作开辟了一种高度并行化的蒙特卡洛推理策略设计空间，其优于标准的best-of-N采样，无需微调，且可由现有语言模型自动实现。
原文摘要 · Abstract (English)
While test-time reasoning enables language models (LMs) to tackle complex tasks, searching or planning in natural language can be slow, costly, and error-prone. But even when LMs struggle to emulate the precise reasoning steps needed to solve a problem, they often excel at describing its abstract structure--both how to verify solutions and how to search for them. This paper introduces DisCIPL, a method for "self-steering" LMs where a Planner model generates a task-specific inference program that is executed by a population of Follower models. Our approach equips LMs with the ability to write recursive search procedures that guide LM inference, enabling new forms of verifiable and efficient reasoning. When instantiated with a small Follower (e.g., Llama-3.2-1B or Qwen3-1.7B), DisCIPL matches (and sometimes outperforms) much larger models, including GPT-4o and o1, on challenging constrained generation tasks. Our work opens up a design space of highly-parallelized Monte Carlo inference strategies that outperform standard best-of-N sampling, require no finetuning, and can be implemented automatically by existing LMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。