让小模型提前判断难题,智能分流大模型任务,提速推理不丢精度。
Before Thinking, Learn to Decide: Proactive Routing for Efficient Visual Reasoning

- 小模型自建信心评分,大模型预判解题能力,双评估早做分流决策。
- 在多个多模态推理基准上,推理速度提升2.1倍以上,准确率保持不变。
- 适合追求高效视觉推理的开发者,尤其对资源受限场景友好。
大型多模态模型在复杂视觉任务上表现出强大推理能力,但其推理效率常受限于过长的思维链。一种有前景的解决方案是将小型草稿模型与大型目标模型配对,通过路由信号自适应地将查询分配给草稿模型或目标模型,以实现最佳效率与准确率的平衡。然而,当前瓶颈在于如何在多模态环境下建立可靠的查询难度信号。现有方法针对语言模型设计,要么依赖事后词元概率(在多模态场景中表现不佳),要么依赖监督微调(数据敏感)。两者均在输出完成后才进行路由,且忽视目标模型实际能否解决被分发的实例。为此,我们提出PRP(Proactive Routing Paradigm),通过联合评估草稿模型和目标模型的能力实现早期决策。草稿评分学习(DRL)赋予草稿模型内部置信度估计器,联合评分学习(JRL)预测目标模型处理特定查询的能力,从而优先分配擅长的任务而非最难的样本。这些评分支持细粒度、实例级的主动路由,显著加速推理而不牺牲整体性能。在多个多模态推理基准上的大量实验验证了该方法的有效性与高效性。
原文摘要 · Abstract (English)
Large multimodal models have achieved strong reasoning on complex visual tasks, but their inference efficiency is often restricted by long chains of thought. A promising solution is to pair a small draft model with a large target model, enabling cooperative inference employing a routing signal that adaptively routes queries to either the draft or target model based on their difficulties for optimal efficiency and accuracy. Yet, the remaining bottleneck is to establish a reliable query difficulty signal under multimodal settings. Existing approaches designed for language models either rely on post-hoc token probabilities, which fall short in multimodal scenarios, or depend on supervised fine-tuning, which is a data-sensitive strategy. Both paradigms perform routing only after a complete output, and ignore whether the target model can actually solve the routed instances. To address this, we propose PRP, a Proactive Routing Paradigm that enables early decision-making by jointly evaluating the competence of both the draft and target models. Our Draft Rating Learning (DRL) equips the draft model with an internal confidence estimator, while Joint Rating Learning (JRL) predicts how well the target model can handle a given query, thereby prioritizing the allocation of samples it excels at rather than the hardest ones. These ratings enable fine-grained, instance-level \textbf{Proactive Routing} and substantially accelerate inference without compromising overall performance. Extensive experiments across multiple multimodal reasoning benchmarks validate our effectiveness and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。