用情感特征提升语音深度伪造检测,不破坏原有语音表示能力
The Affective Bridge: Preserving Speech Representations while Enhancing Deepfake Detection vian emotional Constraints
- 仅用情感识别任务微调语音编码器,再用轻量SVM做检测
- 在FakeOrReal上检测准确率提升6%,在In-the-Wild上提升2%
- 情感是连接语音表示与检测任务的关键桥梁,适合多任务语音系统
语音深度伪造检测(DFD)受益于多种声学与语义语音表示,这些表示包含重要语音信息且训练成本高。已有研究表明情感线索可提升DFD性能,但现有方法或通过复杂流水线融合情感与其他特定任务特征,或直接对表示进行针对DFD的微调,可能扭曲原始表示,影响下游任务如说话人验证(SV)或自动语音识别(ASR)。本文提出更简洁方案:仅在情感识别任务上微调语音编码器,不引入任何DFD监督;随后在冻结的情感调优表示上训练轻量级支持向量机(SVM)进行DFD。该方法保留了原始表示对下游任务的支持能力,同时显著提升DFD性能。关键发现:情感作为桥接任务具有独特优势,替换为说话人身份反而降低检测性能,证明其有效性源于情感在语音表示与检测间的天然桥梁作用。在FakeOrReal和In-the-Wild数据集上,准确率分别提升6%和2%,对应错误等误率(EER)下降;在ASVspoof 2019 LA分析中发现真实语音子集存在数据集特异性说话人偏差。代码见补充材料。
原文摘要 · Abstract (English)
Speech deepfake detection (DFD) has benefited from diverse acoustic and semantic speech representations, many of which encode valuable speech information and are costly to train. Prior work has shown that affective cues improve DFD, yet existing approaches either fuse emotion with other task-specific features in complex pipelines or directly fine-tune representations toward DFD objectives, risking distortion of the original speech representations that support downstream tasks such as speaker verification (SV) or automatic speech recognition (ASR). We propose a simpler approach: fine-tuning speech encoders on emotion recognition alone-without any DFD supervision, and training a lightweight support vector machine (SVM) on the frozen emotion-tuned representations for DFD. This preserves the original representation capacity for downstream tasks such as SV and ASR, while emergently improving DFD performance. Crucially, we find that emotion is uniquely effective as this bridging task: replacing it with speaker identity even degrades DFD performance, demonstrating that the benefit stems from emotion's role as a natural bridge between speech representation and DFD. Experiments on FakeOrReal and In-the-Wild show accuracy improvements of up to 6\% and 2\% with corresponding EER reductions, while analysis on ASVspoof 2019 LA reveals dataset-specific speaker bias in the real-speech subset. Code is available at supplementary materials.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。