通过连续推测加速多跳检索，不改变最终结果却显著降低延迟。
SpecHop: Continuous Speculation for Accelerating Multi-Hop Retrieval Agents
- 维护多个推测线程，异步验证预测结果
- 实测延迟最高降低40%，接近理想最优值
- 适合需要快速响应的复杂信息检索场景
大语言模型越来越多地依赖外部工具（如网络搜索、文档检索）解决信息密集型任务。然而，在复杂任务中进行多跳工具调用会引入显著延迟，因为模型必须反复等待工具返回结果才能继续。本文研究如何在不改变原轨迹的前提下加速此类过程，假设存在更快但可靠性较低的推测工具。我们建立了多跳工具使用场景下的无损推测理论框架，刻画了可实现的最佳延迟收益。提出SpecHop：一种连续推测框架，维持多个推测线程，异步验证预测观察结果，确认正确分支并回滚错误分支，从而在保持准确性的前提下减少实际运行时间。实验表明，SpecHop在检索增强型多跳任务上能逼近理论预测性能，部分场景延迟降低达40%。
原文摘要 · Abstract (English)
Large language models increasingly use external tools such as web search and document retrieval to solve information-intensive tasks. However, multi-hop tool use in complex tasks introduces substantial latency, since the model must repeatedly wait for tool observations before continuing. We study how to accelerate such trajectories without changing the final trajectory the model would have taken without acceleration, assuming access to faster but less reliable speculator tools. We develop a theoretical framework for lossless speculation in multi-hop tool-use settings, characterizing the optimal achievable latency gain. We propose SpecHop, a continuous speculation framework that maintains multiple speculative threads, verifies predicted observations asynchronously as target tool outputs arrive, commits correct branches, and rolls back incorrect ones. This preserves accuracy while reducing wall-clock latency. We show that SpecHop can approach oracle latency gains with enough active threads. Empirically, on retrieval-augmented multi-hop tasks, SpecHop closely matches theoretical predictions and reduces latency by up to 40\% in some settings. Code: https://github.com/mehrdadsaberi/spechop
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。