让视觉语言模型自适应处理任意分辨率图像，提升感知精度。
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
- 动态分辨率机制按图像复杂度分配视觉令牌数量。
- 720亿参数版本在多模态评测中媲美GPT-4o和Claude3.5-Sonnet。
- 支持图像视频统一处理，适合多模态任务研发人员。
我们提出Qwen2-VL系列，对前代Qwen-VL模型进行重大升级，打破传统固定分辨率的视觉处理范式。Qwen2-VL引入朴素动态分辨率机制（Naive Dynamic Resolution），使模型能根据输入图像分辨率动态生成不同数量的视觉令牌，提升视觉表征效率与准确性，更贴近人类感知过程。模型融合多模态旋转位置编码（M-RoPE），有效对齐文本、图像与视频中的位置信息，并采用统一框架处理图像与视频，增强视觉理解能力。通过扩大模型规模（2B、8B、72B参数）与训练数据量，研究大型多模态模型的缩放规律。其中，Qwen2-VL-72B在多个多模态基准测试中表现优异，性能可比肩GPT-4o与Claude3.5-Sonnet，优于其他通用型模型。代码已开源：https://github.com/QwenLM/Qwen2-VL。
原文摘要 · Abstract (English)
We present the Qwen2-VL Series, an advanced upgrade of the previous Qwen-VL models that redefines the conventional predetermined-resolution approach in visual processing. Qwen2-VL introduces the Naive Dynamic Resolution mechanism, which enables the model to dynamically process images of varying resolutions into different numbers of visual tokens. This approach allows the model to generate more efficient and accurate visual representations, closely aligning with human perceptual processes. The model also integrates Multimodal Rotary Position Embedding (M-RoPE), facilitating the effective fusion of positional information across text, images, and videos. We employ a unified paradigm for processing both images and videos, enhancing the model's visual perception capabilities. To explore the potential of large multimodal models, Qwen2-VL investigates the scaling laws for large vision-language models (LVLMs). By scaling both the model size-with versions at 2B, 8B, and 72B parameters-and the amount of training data, the Qwen2-VL Series achieves highly competitive performance. Notably, the Qwen2-VL-72B model achieves results comparable to leading models such as GPT-4o and Claude3.5-Sonnet across various multimodal benchmarks, outperforming other generalist models. Code is available at https://github.com/QwenLM/Qwen2-VL .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。