Ministral 3系列轻量级大模型，三尺寸支持推理与指令微调。
Ministral 3
- 通过级联蒸馏迭代剪枝与持续训练构建高效模型
- 3B/8B/14B三尺寸，均含基础、指令、推理三种变体
- 支持图像理解，开源免费，适合资源受限场景
我们提出Ministral 3系列，是一组面向计算与内存受限场景的参数高效密集语言模型，包含3B、8B和14B三种参数规模。每种规模均提供三种变体：通用预训练基础模型、指令微调版及复杂问题求解的推理模型。此外，我们介绍了通过级联蒸馏（Cascade Distillation）技术构建Ministral 3模型的方法，该方法结合迭代剪枝与带蒸馏的持续训练。所有模型均具备图像理解能力，并在Apache 2.0许可下开源。
原文摘要 · Abstract (English)
We introduce the Ministral 3 series, a family of parameter-efficient dense language models designed for compute and memory constrained applications, available in three model sizes: 3B, 8B, and 14B parameters. For each model size, we release three variants: a pretrained base model for general-purpose use, an instruction finetuned, and a reasoning model for complex problem-solving. In addition, we present our recipe to derive the Ministral 3 models through Cascade Distillation, an iterative pruning and continued training with distillation technique. Each model comes with image understanding capabilities, all under the Apache 2.0 license.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。