轻量级MoE模型Compass-V2专为东南亚语言和电商场景优化，兼顾性能与推理成本。
Compass-V2 Technical Report
- 采用300亿参数、仅50亿激活参数的MoE架构，分层设计细粒度与共享专家模块。
- 构建超百亿词的电商语料库与高质量东南亚多语言数据集，提升领域适配性。
- 首创统一框架内融合快速与深度推理，适合追求效率与精准的电商应用开发者。
主流大模型侧重高资源语言，忽视东南亚（SEA）等低资源语言，且通用性强而缺乏电商针对性。为此，我们提出Compass-V2，一种专为东南亚语言和电商场景设计的轻量级混合专家（MoE）模型。模型总参数量300亿，活跃参数仅50亿，结合细粒度与共享专家模块以平衡性能与推理开销。为提升多语言表现，我们构建了迄今最高质量的行业级东南亚语料数据集；为增强电商能力，整合外部数据挖掘与平台内部数据，建成包含数百亿词的专用数据集。此外，首创统一框架下的混合推理机制，同时支持快速思考与深度推理，突破传统需部署双模型的局限。大量实验表明，Compass-V2在子300亿参数模型中，于东南亚多语言及电商任务上达到顶尖水平，同时显著降低推理成本。
原文摘要 · Abstract (English)
Predominant LLMs focus on high-resource languages while leaving low-resource languages, particularly those in Southeast Asia (SEA), underrepresented. In addition, those models are general-purpose and pay limited attention to the e-commerce domain. To overcome these limitations, we introduce Compass-v2, a lightweight Mixture-of-Experts (MoE) model specifically designed for Southeast Asian languages and e-commerce applications. To balance model performance and inference cost, the model is designed with 30B total parameters and 5B active parameters, incorporating both fine-grained and shared expert modules. To enhance multilingual performance, we curated and constructed a high-quality, industry-leading SEA dataset, to the best of our knowledge. To boost performance in the e-commerce domain, we built a dataset comprising hundreds of billions of tokens, sourced through external data mining and internal platform collection. Besides, we pioneered a hybrid reasoning model that supports both fast thinking and deep thinking within a unified framework to enhance the reasoning capabilities, diverging from the conventional industry practice of deploying two separate models. Through extensive experimental evaluations, our model demonstrates state-of-the-art SEA multilingual and e-commerce performance among sub-30B models, while maintaining significantly lower inference cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。