优化视觉语言模型推理效率，显著提速且保持精度。
Empirical Recipes for Efficient and Compact Vision-Language Models
- 通过实证分析定位推理瓶颈，提出针对性优化方案。
- 在InternVL3-2B上首字延迟降低53%，SmolVLM-256M上降低93%。
- 适配多种架构与部署框架，适合资源受限场景使用。
在资源受限环境中部署视觉语言模型（VLMs）需要低延迟和高吞吐量，但现有紧凑型VLMs的推理速度常低于其参数量所暗示的水平。为解释这一差距，我们开展端到端的效率实证分析，系统性地对推理过程进行剖析，识别出主要瓶颈。基于此，我们提出专为紧凑型VLM设计的优化方案，显著降低延迟同时保持准确率。这些技术使InternVL3-2B的首次生成时间（TTFT）减少53%，SmolVLM-256M减少93%。所提方法广泛适用于不同VLM架构和常见部署框架，为构建高效VLM系统提供实用指导。此外，我们研究了如何扩展紧凑型VLM的结构化感知输出，并引入新模型家族ArgusVLM。在多个基准测试中，ArgusVLM展现出强性能，同时保持紧凑高效的特性。
原文摘要 · Abstract (English)
Deploying vision-language models (VLMs) in resource-constrained settings demands low latency and high throughput, yet existing compact VLMs often fall short of the inference speedups their smaller parameter counts suggest. To explain this discrepancy, we conduct an empirical end-to-end efficiency analysis and systematically profile inference to identify the dominant bottlenecks. Based on these findings, we develop optimization recipes tailored to compact VLMs that substantially reduce latency while preserving accuracy. These techniques cut time to first token (TTFT) by 53% on InternVL3-2B and by 93% on SmolVLM-256M. Our recipes are broadly applicable across both VLM architectures and common serving frameworks, providing practical guidance for building efficient VLM systems. Beyond efficiency, we study how to extend compact VLMs with structured perception outputs and introduce the resulting model family, ArgusVLM. Across diverse benchmarks, ArgusVLM achieves strong performance while maintaining a compact and efficient design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。