30B参数的开源混合专家模型,专为德语和英语优化,推理快且高效。
A Sovereign, Open-Source Foundation Model for German and English

- 采用混合专家架构,每令牌仅激活30亿参数,长文本处理更快。
- 在英德双语基准测试中表现超17个开源模型,代码生成能力最强。
- 基于德国本土算力平台训练,数据与代码完全开源,可信赖可复现。
我们提出 Soofi S 30B-A3B,一个面向德语和英语的主权、开源混合专家(MoE)混合架构基础模型。其混合设计每令牌仅激活300亿参数中的30亿,且随上下文增长保持近似恒定的推理缓存,相比密集模型在长文本、高并发场景下具备显著吞吐优势。在约27万亿令牌上预训练,刻意提升德语权重;在英德综合基准上达到14至270亿参数密集模型水平,代码任务表现超越17个开源基线模型,在所有完全开源模型中取得最高英德评测得分,领先于Olmo 3 32B与Apertus 70B。模型由德电在慕尼黑运营的主权级工业人工智能云平台端到端构建。发布将采用高度宽松的开放条款:提供权重、部分中间检查点、完整数据来源记录、超参数及训练评估代码。允许情况下,数据构建工具按宽松许可发布;商业授权数据则以聚合统计与精确混合比例文档化披露。
原文摘要 · Abstract (English)
We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment. Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14 to 27B models on aggregate English and German benchmarks while achieving the best code aggregates in both languages among 17 open base models, and outperforms every European sovereign baseline in our comparison, including ones far larger in active parameters. Among fully open models, Soofi S obtains the highest English and German evaluation scores, ahead of Olmo 3 32B and Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPC scale AI infrastructure operated by Deutsche Telekom in Munich. Soofi S will be released under highly permissive, open-access terms: weights, selected intermediate checkpoints, full per-source data accounting, hyperparameters, and training and evaluation code. Where source licenses permit, data-construction artifacts are released under permissive licenses; commercially licensed sources are documented with aggregate statistics and exact mixture accounting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。