两款专家混合模型，专为长序列编程代理任务设计，支持高效推理。
Laguna M.1/XS.2 Technical Report
- 采用专家混合架构，按需激活参数以提升效率。
- 在多个编程代理基准上表现媲美同规模顶尖开源模型。
- 开源轻量版模型适用于资源受限场景的代码生成任务。
我们提出 Laguna M.1 与 Laguna XS.2 两款基于专家混合（Mixture-of-Experts）的通用基础模型，专为长时序、代理式编程任务设计。M.1 总参数量达 2258 亿（每令牌激活 234 亿），XS.2 总参数量为 334 亿（每令牌激活 30 亿）。两模型均从零开始在统一的内部系统「Model Factory」中端到端训练，该系统集成版本化数据、训练、评估与推理组件，使模型开发实现工业化流程。本文详述 Model Factory 的设计原则与训练全流程，涵盖预训练数据、模型架构、后训练阶段、评估及量化过程。在代理式软件工程与终端任务基准（SWE-bench Verified、SWE-bench Multilingual、SWE-Bench Pro 与 Terminal-Bench 2.0）上，两者性能均达到其参数量级下顶尖开源模型水平。Laguna XS.2 权重已通过 Apache 2.0 协议开源，地址为 https://huggingface.co/collections/poolside/laguna-xs2。
原文摘要 · Abstract (English)
We present Laguna M.1 and Laguna XS.2, two Mixture-of-Experts foundation models built for long-horizon, agentic coding: M.1 has $225.8$B total parameters ($23.4$B activated per token) and XS.2 has $33.4$B total ($3$B activated). Both models were trained from scratch end-to-end inside the same internal system that we refer to as our Model Factory: a tightly-integrated stack of versioned data, training, evaluation, and inference components that turn model development into an industrial process. We describe the principles and design choices of the Model Factory and also detail the end-to-end training process of our models, throughout pre-training data and architecture, post-training stages, evaluation, and quantization. On agentic software engineering and terminal benchmarks (SWE-bench Verified, SWE-bench Multilingual, SWE-Bench Pro, and Terminal-Bench 2.0) M.1 and XS.2 are competitive with state-of-the-art open models in their respective weight classes. Laguna XS.2 weights are released under Apache~2.0 at https://huggingface.co/collections/poolside/laguna-xs2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。