大模型少输出更高效，用多智能体融合小模型推理提升性能
Rethinking Model Efficiency: Multi-Agent Inference with Large Models
- 大模型用短输出+小模型关键推理token，实现高效推理
- 实测表明：大模型用较少输出即可达小模型性能水平
- 适合追求低延迟的视觉语言模型部署场景
多数视觉语言模型（VLM）使用大语言模型（LLM）作为解码器，通过自回归逐词生成响应，输出词数常成为端到端延迟的瓶颈。本文在模拟数据上全面分析了VLM各组件的延迟，发现输出词数少的大模型比输出序列长的小模型更高效。真实世界基准测试验证了这一现象：大模型仅需极少输出词即可达到或超过小模型的性能。为此，我们提出一种多智能体推理框架，让大模型保持短输出，必要时复用小模型的关键推理词。基准任务对比显示，通过重用小模型推理词，可逼近大模型独立推理的效果，验证了方法的有效性。
原文摘要 · Abstract (English)
Most vision-language models (VLMs) apply a large language model (LLM) as the decoder, where the response tokens are generated sequentially through autoregression. Therefore, the number of output tokens can be the bottleneck of the end-to-end latency. However, different models may require vastly different numbers of output tokens to achieve comparable performance. In this work, we conduct a comprehensive analysis of the latency across different components of VLMs on simulated data. The experiment shows that a large model with fewer output tokens can be more efficient than a small model with a long output sequence. The empirical study on diverse real-world benchmarks confirms the observation that a large model can achieve better or comparable performance as a small model with significantly fewer output tokens. To leverage the efficiency of large models, we propose a multi-agent inference framework that keeps large models with short responses but transfers the key reasoning tokens from the small model when necessary. The comparison on benchmark tasks demonstrates that by reusing the reasoning tokens from small models, it can help approach the performance of a large model with its own reasoning, which confirms the effectiveness of our proposal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。