针对哈萨克-俄语双语环境，构建首个专用安全评估数据集Qorgau。
Qorgau: Evaluating LLM Safety in Kazakh-Russian Bilingual Contexts
- 构建哈萨克语与俄语双语安全评估数据集Qorgau，覆盖本地化风险场景。
- 多语言与单语模型在安全表现上差异显著，低资源语言哈萨克语风险更高。
- 适用于关注中亚地区AI安全、双语模型评估的研究者与开发者。
大型语言模型（LLMs）可能生成有害内容，带来用户风险。尽管已有大量研究建立风险分类体系与安全评估提示，但多数集中在英语等单一语言环境，忽视了双语语境下的语言与区域特异性风险，且核心结论在单语与双语设置中可能不同。本文提出Qorgau，一个专为哈萨克斯坦双语环境设计的新型数据集，涵盖哈萨克语（低资源语言）与俄语（高资源语言）。对多语言及语言特定模型的实验表明，安全表现存在显著差异，凸显在类似哈萨克斯坦的国家部署大模型时，需使用定制化的区域安全评估数据集以保障负责任的应用。警告：本论文包含可能具有冒犯性、有害或偏见的内容。
原文摘要 · Abstract (English)
Large language models (LLMs) are known to have the potential to generate harmful content, posing risks to users. While significant progress has been made in developing taxonomies for LLM risks and safety evaluation prompts, most studies have focused on monolingual contexts, primarily in English. However, language- and region-specific risks in bilingual contexts are often overlooked, and core findings can diverge from those in monolingual settings. In this paper, we introduce Qorgau, a novel dataset specifically designed for safety evaluation in Kazakh and Russian, reflecting the unique bilingual context in Kazakhstan, where both Kazakh (a low-resource language) and Russian (a high-resource language) are spoken. Experiments with both multilingual and language-specific LLMs reveal notable differences in safety performance, emphasizing the need for tailored, region-specific datasets to ensure the responsible and safe deployment of LLMs in countries like Kazakhstan. Warning: this paper contains example data that may be offensive, harmful, or biased.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。