10B参数欧洲开源推理模型，上下文达128K，推理效率高且性能强。
Domyn-Small: A European 10B Reasoning Language Model
- 先预训练后分阶段微调，支持32K原生上下文并扩展至128K
- 在数学、代码等任务上表现优异，生成文本量仅为同类模型1/3至1/2
- 开源权重与训练流程，适合研究者和高效推理部署场景
我们推出Domyn-Small，一款100亿参数的开源推理语言模型，采用MIT许可证发布。该模型基于9万亿标记的多语言数据进行初始预训练，随后通过持续预训练（CPT）将原生上下文窗口翻倍至32K，再经数学导向的渐进式监督微调（SFT）。最终的强化学习阶段包含可验证奖励的GRPO、DPO及覆盖数学、代码、多项选择题、指令遵循和工具调用五个领域的大规模多环境GRPO。推理时通过YaRN将上下文扩展至128K，并支持双模式推理的聊天模板切换。在7-10B级模型中（如Qwen3.5-9B、OLMo-3-7B-Think、Nemotron-Nano-8B、Ministral-3-8B），Domyn-Small以约三分之一的生成Token数优于Qwen3.5-9B，仅使用约35%的OLMo-3-7B-Think Token预算，同时达到79.9的IFEval得分和50.0的GPQA-Diamond科学推理成绩。我们同步发布了模型权重与后训练方案，以及用于高性能计算集群的开源框架Domyn Swarm（Apache 2.0）。
原文摘要 · Abstract (English)
We introduce Domyn-Small, a 10-billion-parameter open-weight reasoning language model released under the MIT license. Domyn-Small is the product of an initial pre-training phase on 9 trillion tokens multilingual data, followed by a post-training pipeline for reasoning, instruction following, and context extension. For the latter, we performed a Continued Pre-Training (CPT) phase that doubles the native context window to 32K tokens, followed by SFT with a math-focused annealing run. Finally, the RL phase includes GRPO with verifiable rewards, DPO, and a multi-environment GRPO stage spanning five task domains: mathematics, code, multiple-choice QA, instruction-following, and tool calling. The 32K-token native context extends to 128K at inference via YaRN, and a chat-template toggle enables dual-mode reasoning. Against peer models in the 7--10B class (Qwen3.5-9B, OLMo-3-7B-Think, Nemotron-Nano-8B, Ministral-3-8B), Domyn-Small achieves a strong accuracy-efficiency balance: it produces roughly one-third as many tokens as Qwen3.5-9B and approximately 35% of OLMo-3-7B-Think's token budget on core reasoning benchmarks, while delivering strong instruction-following (IFEval 79.9) and competitive science reasoning (GPQA-Diamond 50.0). We release the weights and the post-training recipe alongside Domyn Swarm (Apache~2.0), an open-source framework for scalable LLM inference on HPC clusters developed during this program and used throughout this work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。