DeepSeek-VL2用专家混合模型提升多模态理解，支持高分辨率图像和高效推理。
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
- 采用动态分块视觉编码与潜空间注意力机制，兼顾高分辨图像处理与推理效率。
- 在图文问答、表格图表理解等任务中表现优异，4.5B激活参数达顶尖水平。
- 适合需要高效多模态理解的开发者，开源代码与模型可免费获取。
我们提出 DeepSeek-VL2，一系列先进的大容量混合专家（MoE）视觉语言模型，相较于前代 DeepSeek-VL 实现了两大核心升级。视觉部分引入适用于不同长宽比高分辨率图像的动态分块编码策略；语言部分采用具备多头潜空间注意力机制的 DeepSeekMoE 模型，将键值缓存压缩为潜向量，实现高效推理与高吞吐。在优化后的多模态数据集上训练后，DeepSeek-VL2 在多项任务中表现出色，涵盖视觉问答、光学字符识别、文档/表格/图表理解及视觉定位等。该系列包含三个版本：DeepSeek-VL2-Tiny（1.0B）、DeepSeek-VL2-Small（2.8B）和 DeepSeek-VL2（4.5B）激活参数。在相似或更少激活参数下，性能媲美或超越现有开源密集与 MoE 模型。代码与预训练模型已在 https://github.com/deepseek-ai/DeepSeek-VL2 公开。
原文摘要 · Abstract (English)
We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeek-VL, through two key major upgrades. For the vision component, we incorporate a dynamic tiling vision encoding strategy designed for processing high-resolution images with different aspect ratios. For the language component, we leverage DeepSeekMoE models with the Multi-head Latent Attention mechanism, which compresses Key-Value cache into latent vectors, to enable efficient inference and high throughput. Trained on an improved vision-language dataset, DeepSeek-VL2 demonstrates superior capabilities across various tasks, including but not limited to visual question answering, optical character recognition, document/table/chart understanding, and visual grounding. Our model series is composed of three variants: DeepSeek-VL2-Tiny, DeepSeek-VL2-Small and DeepSeek-VL2, with 1.0B, 2.8B and 4.5B activated parameters respectively. DeepSeek-VL2 achieves competitive or state-of-the-art performance with similar or fewer activated parameters compared to existing open-source dense and MoE-based models. Codes and pre-trained models are publicly accessible at https://github.com/deepseek-ai/DeepSeek-VL2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。