构建德语-保加利亚语安全评估数据集，揭示多语言LLM安全表现差异。
Schützen: Evaluating LLM Safety in Bulgarian and German Contexts
- 设计跨语言安全数据集Schützen，覆盖德语（高资源）与保加利亚语（低资源）
- 实验发现多语言模型在德语和保加利亚语中安全行为差异显著
- 为德国与保加利亚的LLM负责任部署提供区域化评估工具
大型语言模型正日益应用于专业领域，带来难以预测的风险，包括生成有害或不尊重内容。尽管安全评估数据集已取得显著进展，但现有资源仍以英语和中文为主。这一局限在共享社会文化、法律和伦理背景的语言中尤为突出。为填补这一空白，我们提出Schützen：一个面向德语-保加利亚语的安全评估数据集，用于衡量模型在风险情境下的回应能力，涵盖低资源语言（保加利亚语）和高资源语言（德语）。对多语言及语言特定的LLM进行实验，揭示了跨语言间安全行为的明显差异，凸显了为德国和保加利亚定制区域化评估资源的必要性。数据集与代码见https://github.com/xnlp-lab/Schutzen。警告：本文包含可能令人不适、有害或有偏见的内容。
原文摘要 · Abstract (English)
Large language models are increasingly deployed across professional domains, bringing hard-to-predict risks, including the generation of harmful or disrespectful content. Although substantial progress has been made in developing safety evaluation datasets, existing resources remain overwhelmingly English- and Chinese-centric. This limitation is particularly pronounced when evaluating languages that operate within shared sociocultural, legal, and ethical contexts. To address this gap, we introduce Schützen: a German--Bulgarian safety dataset designed to assess model answerability under risk, covering both a low-resource language (Bulgarian) and a high-resource language (German). Experiments with multilingual and language-specific LLMs reveal pronounced cross-language differences in safety behavior, highlighting the necessity of tailored, region-specific evaluation resources to support the responsible deployment of LLMs in Germany and Bulgaria. Datasets and code are available at https://github.com/xnlp-lab/Schutzen. Warning: this paper contains examples that may be offensive, harmful, or biased.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。