用轻量微调让中文模型说沙特方言，效果远超通用模型。
Saudi-Dialect-ALLaM: LoRA Fine-Tuning for Dialectal Arabic Generation
- 给沙特自研模型加方言标签，实现精准控制
- 方言识别率从47.97%升至84.21%，标准语泄露降至6.21%
- 开源代码与数据表，支持独立验证
阿拉伯语大模型仍以现代标准阿拉伯语为主，对纳吉迪和海伊扎伊等沙特方言支持不足。本文使用自建的5,466对合成指令-响应数据集（50/50比例），对沙特首屈一指的AllaM-7B-Instruct-preview模型进行LoRA微调，探索两种方案：(i) 添加方言标签的训练，(ii) 格式化时省略标签。评估采用外部方言分类器结合文本保真度指标（chrF++、BERTScore）与多样性度量。含标签模型在控制力上表现最佳，沙特方言识别率从47.97%提升至84.21%，标准语泄漏由32.63%降至6.21%，保真度也显著提升（chrF++ +3.53，BERTScore +0.059）。两种微调方案均优于多个强基线模型（Falcon-7B-Instruct、Llama-3.1-8B-Instruct、Qwen-2.5-7B-Instruct、AceGPT-v2-8B-Chat、JAIS-13B-Chat），且避免了基线模型常见的元数据重复问题。本研究不公开数据集或模型权重，但发布训练/评估/推理代码及详细数据说明书（含结构与统计汇总），以支持独立复现。
原文摘要 · Abstract (English)
Large language models (LLMs) for Arabic are still dominated by Modern Standard Arabic (MSA), with limited support for Saudi dialects such as Najdi and Hijazi. This underrepresentation hinders their ability to capture authentic dialectal variation. Using a privately curated Saudi Dialect Instruction dataset (Hijazi and Najdi; 5,466 synthetic instruction-response pairs; 50/50 split), we LoRA-tune ALLaM-7B-Instruct-preview, the first foundation model developed in Saudi Arabia, for Saudi dialect generation. We investigate two variants: (i) Dialect-Token training, which prepends an explicit dialect tag to the instruction, and (ii) No-Token training, which omits the tag at formatting time. Evaluation on a held-out test set combines an external dialect classifier with text fidelity metrics (chrF++ and BERTScore) and diversity measures. The Dialect-Token model achieves the best control, raising the Saudi rate from 47.97% to 84.21% and reducing MSA leakage from 32.63% to 6.21%; fidelity also improves (chrF++ +3.53, BERTScore +0.059). Both LoRA variants outperform strong generic instruction models (Falcon-7B-Instruct, Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct, AceGPT-v2-8B-Chat, JAIS-13B-Chat) in dialect control and fidelity, while avoiding metadata-tag echoing that these baselines frequently exhibit. We do not release the dataset or any model weights/adapters; instead, we release training/evaluation/inference code and a detailed datasheet (schema and aggregate statistics) to support independent verification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。