让低资源语言也能拥有专属大模型，提升多语言能力。
BYOL: Bring Your Own Language Into LLMs
- 按语言数字资源分四类，定制数据与训练路径。
- 对切瓦语和毛利语提升12%基准表现，保留英文能力。
- 极端低资源语言用翻译中介，提升4点BLEU值。
大型语言模型（LLMs）具备强大多语言能力，但受全球语言资源严重不均制约。全球7000多种语言中，仅不到100种有足够数字内容影响现代LLM训练。这种差距导致低资源及极低资源语言使用者在性能、文化适配和可访问性上系统性落后。为此，我们提出「自带语言」（BYOL）框架，根据语言数字足迹将语言分为四类（极低、低、中、高），并匹配相应集成路径。针对低资源语言，设计从语料清洗、合成文本生成、持续预训练到监督微调的全栈扩展流程。应用于切瓦语和毛利语，模型在12个基准上平均比强基线提升约12%，同时通过权重空间融合保留英语与多语言能力。对极低资源语言，提出基于机器翻译的引入路径，在因纽特语上实现4点BLEU提升，使直接建模不可行时仍能获得高精度模型。我们还发布了切瓦语、毛利语和因纽特语版本的Global MMLU-Lite基准，代码与模型已开源于https://github.com/microsoft/byol。
原文摘要 · Abstract (English)
Large Language Models (LLMs) exhibit strong multilingual capabilities, yet remain fundamentally constrained by the severe imbalance in global language resources. While over 7,000 languages are spoken worldwide, only a small subset (fewer than 100) has sufficient digital presence to meaningfully influence modern LLM training. This disparity leads to systematic underperformance, cultural misalignment, and limited accessibility for speakers of low-resource and extreme-low-resource languages. To address this gap, we introduce Bring Your Own Language (BYOL), a unified framework for scalable, language-aware LLM development tailored to each language's digital footprint. BYOL begins with a language resource classification that maps languages into four tiers (Extreme-Low, Low, Mid, High) using curated web-scale corpora, and uses this classification to select the appropriate integration pathway. For low-resource languages, we propose a full-stack data refinement and expansion pipeline that combines corpus cleaning, synthetic text generation, continual pretraining, and supervised finetuning. Applied to Chichewa and Maori, this pipeline yields language-specific LLMs that achieve approximately 12 percent average improvement over strong multilingual baselines across 12 benchmarks, while preserving English and multilingual capabilities via weight-space model merging. For extreme-low-resource languages, we introduce a translation-mediated inclusion pathway, and show on Inuktitut that a tailored machine translation system improves over a commercial baseline by 4 BLEU, enabling high-accuracy LLM access when direct language modeling is infeasible. Finally, we release human-translated versions of the Global MMLU-Lite benchmark in Chichewa, Maori, and Inuktitut, and make our codebase and models publicly available at https://github.com/microsoft/byol .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。