利用命名实体信息提升音频伪造定位精度
NE-PADD: Leveraging Named Entity Knowledge for Robust Partial Audio Deepfake Detection via Attention Aggregation
- 双分支结构融合语音命名实体识别与伪造检测
- 通过注意力融合与迁移机制提升定位准确率
- 适合关注音频安全与细粒度检测的研究者
与传统的句子级音频伪造检测不同，部分音频伪造检测（PADD）需要在帧级别精确定位伪造语音位置。尽管已有一定进展，但利用音频中的语义信息，尤其是命名实体，仍属研究空白。为此，我们提出NE-PADD，一种新型的局部音频伪造检测方法，通过两个并行分支——语音命名实体识别（SpeechNER）和PADD——融合命名实体知识。该方法引入两种注意力聚合机制：注意力融合（AF）用于合并注意力权重，注意力迁移（AT）通过辅助损失将命名实体语义引导至PADD。基于PartialSpoof-NER数据集的实验表明，该方法优于现有基线，验证了融合命名实体知识在PADD中的有效性。代码已开源。
原文摘要 · Abstract (English)
Different from traditional sentence-level audio deepfake detection (ADD), partial audio deepfake detection (PADD) requires frame-level positioning of the location of fake speech. While some progress has been made in this area, leveraging semantic information from audio, especially named entities, remains an underexplored aspect. To this end, we propose NE-PADD, a novel method for Partial Audio Deepfake Detection (PADD) that leverages named entity knowledge through two parallel branches: Speech Name Entity Recognition (SpeechNER) and PADD. The approach incorporates two attention aggregation mechanisms: Attention Fusion (AF) for combining attention weights and Attention Transfer (AT) for guiding PADD with named entity semantics using an auxiliary loss. Built on the PartialSpoof-NER dataset, experiments show our method outperforms existing baselines, proving the effectiveness of integrating named entity knowledge in PADD. The code is available at https://github.com/AI-S2-Lab/NE-PADD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。