通过碎片化推理策略,在减少计算量的同时提升大模型推理效率。
Fractured Chain-of-Thought Reasoning
- 提出碎片采样法,动态控制推理轨迹数量、解的数量和截断深度。
- 在五个基准测试中,相同令牌预算下准确率显著高于完整链式推理。
- 适合对延迟敏感的部署场景,尤其适用于资源受限的大模型应用。
推理时扩展技术通过增加推理阶段的计算量,显著提升了大语言模型(LLM)的推理能力,而无需重新训练。链式思维(CoT)及其扩展长链式思维(Long CoT)通过生成丰富的中间推理路径提高了准确性,但带来了高昂的令牌开销,限制了其在低延迟场景中的应用。本文首先发现,截断式CoT(提前停止推理并直接生成答案)在多数情况下可达到与完整采样相当的准确率,同时大幅减少令牌消耗。基于此,我们提出碎片采样(Fractured Sampling),一种统一的推理时策略,沿三个正交维度进行插值:(1) 推理轨迹数量,(2) 每条轨迹的最终解数量,(3) 推理截断深度。在五个不同推理基准和多个模型规模上的实验表明,该方法在准确率-成本权衡上持续表现更优,实现Pass@k随令牌预算呈陡峭的对数线性增长。分析揭示了如何在各维度间分配计算以最大化性能,为更高效、可扩展的LLM推理铺平道路。代码已开源:https://github.com/BaohaoLiao/frac-cot。
原文摘要 · Abstract (English)
Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference without retraining. Similarly, Chain-of-Thought (CoT) prompting and its extension, Long CoT, improve accuracy by generating rich intermediate reasoning trajectories, but these approaches incur substantial token costs that impede their deployment in latency-sensitive settings. In this work, we first show that truncated CoT, which stops reasoning before completion and directly generates the final answer, often matches the full CoT sampling while using dramatically fewer tokens. Building on this insight, we introduce Fractured Sampling, a unified inference-time strategy that interpolates between full CoT and solution-only sampling along three orthogonal axes: (1) the number of reasoning trajectories, (2) the number of final solutions per trajectory, and (3) the depth at which reasoning traces are truncated. Through extensive experiments on five diverse reasoning benchmarks and several model scales, we demonstrate that Fractured Sampling consistently achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling gains in Pass@k versus token budget. Our analysis reveals how to allocate computation across these dimensions to maximize performance, paving the way for more efficient and scalable LLM reasoning. Code is available at https://github.com/BaohaoLiao/frac-cot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。