用可变资源的模型程序实现高效多模态推理
Resource-efficient Inference with Foundation Model Programs
- 将任务拆解为模块化程序，动态选择不同规模的模型后端
- 在流式视觉问答中实现98%资源节省，精度几乎无损
- 适合追求低成本部署的生产系统和边缘计算场景
大型语言与视觉模型的推理阶段资源消耗日益成为生产部署的挑战。本文提出使用基础模型程序——即能调用不同资源成本与性能的基础模型的程序——来应对该问题。具体而言，我们方法将任务转化为程序，并学习一个资源分配策略：针对每个输入，为程序模块选择合适的模型后端。该策略对简单子任务使用更小、更便宜的模型，复杂子任务则启用更大、更强大的模型。我们在两个新的“流式”视觉问答任务上进行评估，系统需对一系列输入序列回答问题，并在每次回答后获得真实反馈。相比单体多模态模型，我们的实现最多节省98%资源，同时保持极小精度损失，展示了其在可扩展、资源高效多模态推理中的潜力。
原文摘要 · Abstract (English)
The inference-time resource costs of large language and vision models present a growing challenge in production deployments. We propose the use of foundation model programs, i.e., programs that can invoke foundation models with varying resource costs and performance, as an approach to this problem. Specifically, we present a method that translates a task into a program, then learns a policy for resource allocation that, on each input, selects foundation model "backends" for each program module. The policy uses smaller, cheaper backends to handle simpler subtasks, while allowing more complex subtasks to leverage larger, more capable models. We evaluate the method on two new "streaming" visual question-answering tasks in which a system answers a question on a sequence of inputs, receiving ground-truth feedback after each answer. Compared to monolithic multi-modal models, our implementation achieves up to 98% resource savings with minimal accuracy loss, demonstrating its potential for scalable and resource-efficient multi-modal inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。