通过早期投票与轨迹剪枝，让大模型推理更高效且不丢精度。
Funnel of Thoughts: Efficient Test-Time Scaling via Early Voting and Rollout Pruning
- 利用重复犹豫词识别无效推理路径，提前剪枝
- 在保持32条轨迹投票准确率前提下，减少56.1%注意力计算量
- 无需训练，跨模型、跨任务通用，适合高并发推理场景
大型推理模型在相同问题上多次运行会产生多样甚至不一致的答案，因此多样本推理是可靠部署的必要条件。主流做法是在k次轨迹中进行多数投票，但代价高昂。本文提出无监督的“思想漏斗”（FoT）方法，在维持32条轨迹投票准确率的同时，将注意力计算量（FLOPs）减半，全模型推理成本降低28.8%。通过对六种大模型共11.5万条推理轨迹分析发现，无效轨迹常表现为反复出现的犹豫词汇如"Wait"、"Actually"、"perhaps"，这些轨迹难以得出正确答案，且消耗大量注意力资源，极端情况下陷入无解循环。基于此无需训练的词汇信号，FoT可提前识别并剪枝低效轨迹，使在线生成注意力计算量下降56.1%，耗时减少37.6%，且无需额外模型推理；该信号在未见架构和域外任务中亦有效迁移。
原文摘要 · Abstract (English)
Large Reasoning Models produce diverse, sometimes inconsistent answers across repeated queries on the same problem, so multi-sample inference is a prerequisite for reliable deployment. Majority voting at k rollouts is the standard solution and the de facto accuracy target for this regime, but it is prohibitively expensive at the scale LRMs require. We introduce Funnel of Thoughts (FoT), an inference-time method that preserves the full 32-trajectory voted accuracy while halving its attention FLOPs, a 28.8% reduction in full-model inference cost. Across 115K reasoning trajectories from six LRMs, we find that unproductive trajectories often reveal themselves through repeated hesitation markers such as "Wait", "Actually", and "perhaps." These trajectories are less likely to reach the correct answer and consume disproportionate attention FLOPs, degenerating into no-answer loops in the worst case. Built on this training-free lexical signal, FoT identifies the vocabulary that captures these pathological patterns and prunes affected trajectories before completion, reducing online generation attention FLOPs by 56.1% and wall time by 37.6% without any additional model inference; the same signal transfers without retuning across held-out architectures and out-of-domain tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。