用少量高质量数据让大模型流畅说摩洛哥方言，同时不丢推理能力。
GemMaroc: Unlocking Darija Proficiency in LLMs with Minimal Data
- 精选3个指令集翻译成方言，结合数学编码题微调模型。
- 4千条混合指令使方言测评分提升至47.5，27B版达61.6分。
- 训练仅需48GPU小时，适合教育与公共服务场景应用。
开源大语言模型仍忽视摩洛哥阿拉伯语（Darija），导致使用者要么使用重型阿拉伯语适配器，要么牺牲模型的推理能力。本文提出一种质量优先的对齐策略，仅用极少计算资源即可激活模型的流利方言表达能力，并保持跨语言推理性能。将三个小型指令集LIMA 1K、DEITA 6K和TULU 50K翻译为Darija，保留20条英文原指令，并新增数学、编程与科学类提示。在5千条混合指令上微调Gemma 3-4B模型，使DarijaMMLU得分从32.8提升至47.5，且无英语能力退化。同方法扩展至Gemma 3-27B，生成GemMaroc-27B，在DarijaMMLU上达到61.6分，超越Atlas-Chat；在HellaSwag常识测试中达60.5分，高于Atlas-Chat的48.4。关键的是，该模型在GSM8K及英语基准测试中仅出现轻微波动。整个训练过程仅耗时48 GPU小时，体现绿色人工智能路径。代码、数据与模型权重已公开，推动方言在教育、公共服务与日常数字交互中的应用。
原文摘要 · Abstract (English)
Open-source large language models (LLMs) still marginalise Moroccan Arabic (Darija), forcing practitioners either to bolt on heavyweight Arabic adapters or to sacrifice the very reasoning skills that make LLMs useful. We show that a rigorously quality-over-quantity alignment strategy can surface fluent Darija while safeguarding the backbone s cross-lingual reasoning at a sliver of the usual compute. We translate three compact instruction suites LIMA 1 K, DEITA 6 K and TULU 50 K into Darija, preserve 20 of the English originals, and add mathematics, coding and scientific prompts. A LoRA-tuned Gemma 3-4B trained on 5 K mixed instructions lifts DarijaMMLU from 32.8 to 42.7 ; adding the reasoning-dense TULU portion pushes it to 47.5 with no English regression. Scaling the identical recipe to Gemma 3-27B produces GemMaroc-27B, which matches Atlas-Chat on DarijaMMLU (61.6 ) and leaps ahead on Darija commonsense, scoring 60.5 on HellaSwag versus Atlas-Chat s 48.4 . Crucially, GemMaroc retains Gemma-27B s strong maths and general-reasoning ability, showing only minimal movement on GSM8K and English benchmarks. The entire model is trained in just 48 GPU.h, underscoring a Green AI pathway to inclusive, sustainable language technology. We release code, data and checkpoints to spur Darija-centric applications in education, public services and everyday digital interaction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。