小模型实现强视觉理解，超越多个大模型。
ZAYA1-VL-8B Technical Report
- 用视觉专用LoRA增强语言模型，不增加专家数量
- 在图像理解、推理与计数任务上表现优异，参数仅9.2B
- 适合资源有限但需强多模态能力的场景
我们提出ZAYA1-VL-8B，一个基于自研语言模型ZAYA1-8B构建的紧凑型混合专家视觉语言模型。尽管规模较小，其性能可媲美领先的基础模型如Molmo2-4B和InternVL3.5-4B，且在多项图像理解、推理与计数基准测试中超越Qwen2.5-VL-3B、PLM-3B和MolmoE-1B。该模型架构包含两项关键创新：（1）在语言模型中集成视觉专用LoRA适配器，提升模态特异性能力而不增加专家数量；（2）在语言模型内部对图像标记实施双向注意力机制，增强视觉理解。本文详述完整训练流程，包括各阶段数据构成、序列打包及注意力掩码设计。模型总参数量为9.2B，其中活跃参数1.4B（含视觉编码器），现已公开于https://huggingface.co/Zyphra/ZAYA1-VL。
原文摘要 · Abstract (English)
We present ZAYA1-VL-8B, a compact mixture-of-experts vision-language model built upon our in-house language model, ZAYA1-8B. Despite its compact size, ZAYA1-VL achieves performance competitive with leading base models such as Molmo2-4B and InternVL3.5-4B, while surpassing models including Qwen2.5-VL-3B, PLM-3B, and MolmoE-1B across a range of image understanding, reasoning, and counting benchmarks. The architecture incorporates two key innovations: (1) vision-specific LoRA adapters integrated into the LLM to increase modality-specific capacity without increasing the number of experts, and (2) bidirectional attention over image tokens within the LLM to enhance visual understanding. We detail the full training pipeline including data composition at each stage, sequence packing, and the attention masking scheme. The model comprises 9.2B total parameters, with 1.4B active parameters including the vision encoder, and is publicly available at https://huggingface.co/Zyphra/ZAYA1-VL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。