arXiv:2507.01006cs.CVcs.AI2025-07被引 372

GLM系列多模态模型通过强化学习提升推理能力,开源版本性能媲美甚至超越闭源大模型。

GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning

  • 采用课程采样强化学习,激活视觉基础模型的深层推理潜力。
  • 在42个基准上表现领先,9B小模型胜过72B的Qwen2.5-VL。
  • 支持工具调用与128K上下文,适合复杂任务如编程和长文档理解。

我们推出了GLM-4.1V-Thinking、GLM-4.5V和GLM-4.6V系列多模态模型,致力于推动通用多模态理解和推理。通过大规模预训练构建具备高潜力的视觉基础模型,奠定性能上限。随后提出课程采样强化学习(RLCS)框架,显著提升模型在科学计算、视频理解、内容识别、代码生成、视觉定位、基于GUI的智能体及长文档解析等多样化任务上的综合能力。在42个公开基准的全面评估中,GLM-4.5V在同规模开源模型中接近顶尖水平,在编程和GUI智能体等挑战性任务上甚至优于闭源模型Gemini-2.5-Flash。而更小的GLM-4.1V-9B-Thinking在29个基准上超越更大规模的Qwen2.5-VL-72B。两个模型均已开源。此外,我们还推出GLM-4.6V系列,支持原生工具使用和128K上下文窗口。更多信息详见https://z.ai/blog/glm-4.6v,代码与模型发布于https://github.com/zai-org/GLM-V。

原文摘要 · Abstract (English)

We present GLM-4.1V-Thinking, GLM-4.5V, and GLM-4.6V, a family of vision-language models (VLMs) designed to advance general-purpose multimodal understanding and reasoning. In this report, we share our key findings in the development of the reasoning-centric training framework. We first develop a capable vision foundation model with significant potential through large-scale pre-training, which arguably sets the upper bound for the final performance. We then propose Reinforcement Learning with Curriculum Sampling (RLCS) to unlock the full potential of the model, leading to comprehensive capability enhancement across a diverse range of tasks, including STEM problem solving, video understanding, content recognition, coding, grounding, GUI-based agents, and long document interpretation. In a comprehensive evaluation across 42 public benchmarks, GLM-4.5V achieves state-of-the-art performance on nearly all tasks among open-source models of similar size, and demonstrates competitive or even superior results compared to closed-source models such as Gemini-2.5-Flash on challenging tasks including Coding and GUI Agents. Meanwhile, the smaller GLM-4.1V-9B-Thinking remains highly competitive-achieving superior results to the much larger Qwen2.5-VL-72B on 29 benchmarks. We open-source both GLM-4.1V-9B-Thinking and GLM-4.5V. We further introduce the GLM-4.6V series, open-source multimodal models with native tool use and a 128K context window. A brief overview is available at https://z.ai/blog/glm-4.6v. Code, models and more information are released at https://github.com/zai-org/GLM-V.

多模态推理强化学习开源模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。