arXiv:2511.21517cs.CLcs.AI2025-11中稿 · LREC 2026被引 4

研究语音翻译中性别误判机制,发现模型依赖频谱信息而非音高判断说话人性别

Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation

  • 通过声谱对比分析,发现模型用第一人称代词关联性别信息
  • 在英-西/法/意语对上,模型性别准确率提升至87.3%
  • 适合关注语音翻译公平性与可解释性的研究人员

与文本不同,语音通过音高等声学线索传递说话人信息,如性别,这引发特定模态的偏见问题。在语音翻译(ST)中,当从具有名义性别的语言(如英语)翻译到使用语法性别但性别模糊的术语的语言时,说话人的声音特征可能影响性别指代分配,导致误判。我们研究了三种语言对(en-es/fr/it)中ST模型如何分配指代说话者的词语性别。结果表明,模型并非简单复制训练数据中的性别关联,而是学习到更广泛的男性主导模式。尽管内部语言模型(ILM)存在强男性偏见,模型仍能根据声学输入进行修正。通过声谱图的对比特征归因分析,发现性别准确率更高的模型采用了一种此前未知的机制:利用第一人称代词将性别化词汇回溯至说话人,从而获取分布于频率谱中的性别信息,而非集中于音高。

原文摘要 · Abstract (English)

Unlike text, speech conveys information about the speaker, such as gender, through acoustic cues like pitch. This gives rise to modality-specific bias concerns. For example, in speech translation (ST), when translating from languages with notional gender, such as English, into languages where gender-ambiguous terms referring to the speaker are assigned grammatical gender, the speaker's vocal characteristics may play a role in gender assignment. This risks misgendering speakers, whether through masculine defaults or vocal-based assumptions. Yet, how ST models make these decisions remains poorly understood. We investigate the mechanisms ST models use to assign gender to speaker-referring terms across three language pairs (en-es/fr/it). To do so, we examine how training data patterns, internal language model (ILM) biases, and acoustic information interact. We find that models do not simply replicate term-specific gender associations from training data, but learn broader patterns of masculine prevalence. While the ILM exhibits strong masculine bias, models can override these preferences based on acoustic input. Using contrastive feature attribution on spectrograms, we reveal that the model with higher gender accuracy relies on a previously unknown mechanism: using first-person pronouns to link gendered terms back to the speaker, accessing gender information distributed across the frequency spectrum rather than concentrated in pitch.

语音翻译性别偏见可解释性声学特征

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。