构建多语言中性翻译评估集，发现大模型难稳定生成无性别偏见译文。
Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE
- 设计专家标注的mGeNTE多语言评估集，覆盖英-西/德/意/希语对
- 测试显示大模型能识别需中性表达的场景，但无法稳定生成中性译文
- 通过可解释性分析揭示模型内部决策机制，助力改进公平性
避免不必要（二元）性别推断和默认男性化语言仍是实现包容性多语言技术的关键挑战，尤其是在具有丰富性别形态的语言中。中性翻译（GNT）是一种促进跨语言公平沟通的语言策略。然而，现有GNT研究仅限于少数资源和语对。为此，我们提出mGeNTE——一个专家标注的多语言资源，并首次使用最先进的指令遵循语言模型（LMs）对多语言包容性翻译进行系统性评估。在en-es/de/it/el语对上的实验表明，尽管模型能识别中性适用场景，却无法一致生成中性译文，限制了其实际可用性。为进一步探究该现象，我们引入可解释性分析，识别出任务相关特征，并初步揭示了基于语言模型的中性翻译内部动态。
原文摘要 · Abstract (English)
Avoiding the propagation of undue (binary) gender inferences and default masculine language remains a key challenge towards inclusive multilingual technologies, particularly when translating into languages with extensive gendered morphology. Gender-neutral translation (GNT) represents a linguistic strategy towards fairer communication across languages. However, research on GNT is limited to a few resources and language pairs. To address this gap, we introduce mGeNTE, an expert-curated resource, and use it to conduct the first systematic multilingual evaluation of inclusive translation with state-of-the-art instruction-following language models (LMs). Experiments on en-es/de/it/el reveal that while models can recognize when neutrality is appropriate, they cannot consistently produce neutral translations, limiting their usability. To probe this behavior, we enrich our evaluation with interpretability analyses that identify task-relevant features and offer initial insights into the internal dynamics of LM-based GNT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。