通过负向提前退出与自适应调度，显著降低大模型推理时的延迟高峰。
Adaptive Parallel Monte Carlo Tree Search for Efficient Test-time Compute Scaling
- 引入负向提前退出，剪枝无进展的搜索路径
- 自适应重分配计算资源，缓解并发搜索竞争
- 在保持推理准确率前提下，降低99%尾部延迟
蒙特卡洛树搜索（MCTS）是提升大语言模型推理性能的有效测试时计算扩展（TTCS）方法，但其执行时间高度波动，导致实际应用中出现严重的长尾延迟。现有优化如正向提前退出在有利情况下可降低延迟，但在搜索持续且无明显进展时效果有限。本文提出负向提前退出机制，用于剪枝无产出的MCTS轨迹，并设计自适应增强机制，将释放的计算资源重新分配以减少并发搜索间的资源争用。该方法集成于vLLM框架，显著降低了端到端p99延迟，同时提升吞吐量并保持推理准确性。
原文摘要 · Abstract (English)
Monte Carlo Tree Search (MCTS) is an effective test-time compute scaling (TTCS) method for improving the reasoning performance of large language models, but its highly variable execution time leads to severe long-tail latency in practice. Existing optimizations such as positive early exit, reduce latency in favorable cases but are less effective when search continues without meaningful progress. We introduce {\it negative early exit}, which prunes unproductive MCTS trajectories, and an {\it adaptive boosting mechanism} that reallocates reclaimed computation to reduce resource contention among concurrent searches. Integrated into vLLM, these techniques substantially reduce p99 end-to-end latency while improving throughput and maintaining reasoning accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。