跨模态对齐检测低资源语言深伪仇恨言论，零样本也能准。
Multimodal Zero-Shot Framework for Deepfake Hate Speech Detection in Low-Resource Languages
- 用对比学习对齐多语言音文表征，构建共享语义空间。
- 在六种语言上达0.819和0.701准确率，零样本泛化强。
- 首个针对低资源印地语的深伪仇恨言论数据集，适合安全研究者。
本文提出一种新型多模态框架，用于深伪音频中的仇恨言论检测，尤其在零样本场景表现优异。与以往方法不同，该方法采用对比学习，联合对齐跨语言的音频与文本表征。我们构建了首个包含127,290对文本与合成语音样本的基准数据集，涵盖英语及五种低资源印度语言（印地语、孟加拉语、马拉地语、泰米尔语、泰卢固语）。模型学习共享语义嵌入空间，实现鲁棒的跨语言、跨模态分类。在两个多语言测试集上的实验表明，本方法优于基线，准确率分别达到0.819和0.701，且能良好推广至未见语言。这证明融合多模态在合成媒体仇恨言论检测中的优势，尤其在单模态模型表现薄弱的低资源环境中。数据集已公开：https://www.iab-rubric.org/resources。
原文摘要 · Abstract (English)
This paper introduces a novel multimodal framework for hate speech detection in deepfake audio, excelling even in zero-shot scenarios. Unlike previous approaches, our method uses contrastive learning to jointly align audio and text representations across languages. We present the first benchmark dataset with 127,290 paired text and synthesized speech samples in six languages: English and five low-resource Indian languages (Hindi, Bengali, Marathi, Tamil, Telugu). Our model learns a shared semantic embedding space, enabling robust cross-lingual and cross-modal classification. Experiments on two multilingual test sets show our approach outperforms baselines, achieving accuracies of 0.819 and 0.701, and generalizes well to unseen languages. This demonstrates the advantage of combining modalities for hate speech detection in synthetic media, especially in low-resource settings where unimodal models falter. The Dataset is available at https://www.iab-rubric.org/resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。