20亿参数的轻量级视觉大模型，专为边缘设备实时推理设计。
AuroraEdge-V-2B: A Faster And Stronger Edge Visual Large Language Model
- 仅20亿参数，通过压缩融合技术提升推理速度。
- 视觉令牌减少一半，推理浮点运算量降50%。
- 在9项基准测试中超越同规模模型，适合工业边缘部署。
随着多模态技术的发展，视觉大语言模型（VLLMs）正逐步替代工业生产中的深度学习模型（DLMs）。尽管VLLMs具备强泛化能力与上下文学习灵活性，但在特定领域表现不如定制模型，且参数量大、计算资源需求高、推理速度慢，难以实现实时响应。为此，本文提出AuroraEdge-V-2B，一款专为边缘部署设计的紧凑、鲁棒且高速的视觉大模型。通过提出的压缩融合方法，显著提升推理效率。该模型仅含20亿参数，易于部署且支持实时响应；在解码过程中大幅减少视觉令牌数量，使推理浮点运算量减半，成本更低；在9个基准测试中性能优于同等参数量模型（如Qwen2-VL-2B、Qwen2.5-VL-3B、InternVL-2.5-2B），展现出更强的实用性。
原文摘要 · Abstract (English)
Recently, due to the advancement of multimodal technology, people are attempting to use visual large language models (VLLMs) in industrial production. Many deep learning models (DLMs) deployed in the production environment are gradually being replaced by VLLMs. Compared with DLMs, VLLMs have some advantages in industrial applications: (1) Their strong generalization ability enables them to perform well across a wide range of tasks. (2) They are flexible and can deal with unfamiliar samples through context learning quickly. However, VLLMs also have obvious drawbacks: (1) VLLMs do not perform as well as custom-developed DLMs in specific domains. (2) The number of parameters in VLLMs is generally quite large, and their deployment requires substantial computational resources. (3) VLLMs generally operate much slower than DLMs, making real-time response challenging to achieve. To better utilize VLLMs in industrial applications, we introduce AuroraEdge-V-2B in this work, a compact, robust, and high-speed VLLM designed for edge deployment. To make the model run faster, we also propose a compression-fusion method to improve inference efficiency. AuroraEdge-V-2B has the following notable features: (1) Easy deployment and faster: It has only 2B parameters and is highly suitable for edge deployment, offering better real-time performance. (2) Fewer visual tokens and cheaper: It significantly reduces the number of visual tokens in the decoding process, thereby reducing the floating-point operations by half during inference and making it cheaper to use. (3) Strong performance: It gets a higher score on 9 benchmarks than models with the same number of parameter (e.g., Qwen2-VL-2B, Qwen2.5-VL-3B, InternVL-2.5-2B).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。