用小模型主跑,只让大模型处理关键分歧点,提速2.8倍还更准。
R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing
- 只在少数分歧token上调用大模型,其余由小模型完成生成。
- 平均激活参数仅5.6B,准确率超R1-7B的1.6倍,逼近R1-14B性能。
- 适合追求高效推理的部署场景,尤其数学、编程等复杂任务。
大型语言模型(LLMs)虽具强大推理能力,但推理开销巨大,部署困难。小型语言模型(SLMs)虽效率高,却因无法跟随大模型的推理路径而性能下降。我们发现,只有少量token真正导致推理路径分歧,多数生成结果相同或仅存在表达差异。基于此,提出**道路通罗马(R2R)**,一种神经令牌路由方法:仅对关键分歧令牌调用大模型,其余由小模型生成。我们还构建自动数据生成管道,识别分歧令牌并生成令牌级路由标签以训练轻量路由器。将R1-1.5B与R1-32B模型结合,评估在数学、编程和问答等挑战性基准上的表现。平均激活参数为5.6B时,准确率是R1-7B的1.6倍,超越R1-14B;相比R1-32B,实现2.8倍的实时加速且性能相当,显著提升推理效率的帕累托前沿。代码已开源。
原文摘要 · Abstract (English)
Large Language Models (LLMs) achieve impressive reasoning capabilities at the cost of substantial inference overhead, posing substantial deployment challenges. Although distilled Small Language Models (SLMs) significantly enhance efficiency, their performance suffers as they fail to follow LLMs' reasoning paths. Luckily, we reveal that only a small fraction of tokens genuinely diverge reasoning paths between LLMs and SLMs. Most generated tokens are either identical or exhibit neutral differences, such as minor variations in abbreviations or expressions. Leveraging this insight, we introduce **Roads to Rome (R2R)**, a neural token routing method that selectively utilizes LLMs only for these critical, path-divergent tokens, while leaving the majority of token generation to the SLM. We also develop an automatic data generation pipeline that identifies divergent tokens and generates token-level routing labels to train the lightweight router. We apply R2R to combine R1-1.5B and R1-32B models from the DeepSeek family, and evaluate on challenging math, coding, and QA benchmarks. With an average activated parameter size of 5.6B, R2R surpasses the average accuracy of R1-7B by 1.6x, outperforming even the R1-14B model. Compared to R1-32B, it delivers a 2.8x wall-clock speedup with comparable performance, advancing the Pareto frontier of test-time scaling efficiency. Our code is available at https://github.com/thu-nics/R2R.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。