arXiv:2411.09167cs.SDcs.CR2024-11被引 17

提出新方法检测深度伪造语音,提升对未知合成器的泛化能力。

Robust AI-Synthesized Speech Detection Using Feature Decomposition Learning and Synthesizer Feature Augmentation

  • 分离合成器特征与内容特征,双流学习增强鲁棒性。
  • 通过伪标签和对抗训练,让模型学会忽略合成器痕迹。
  • 随机混合特征风格,模拟更多伪造场景,适合安全与质检领域。

AI生成语音(即深度伪造语音)因语音合成与转换技术的快速发展引发广泛关注。以往方法依赖识别特定合成器的痕迹,但在面对未见过的合成器时性能下降。本文提出一种鲁棒的深度伪造语音检测方法,采用特征分解学习,提取与合成器无关的内容特征作为补充。具体地,设计双流特征分解学习策略:合成器流通过合成器标签监督训练,专注学习合成器特征;内容流则通过伪标签监督学习,利用随机变速与压缩生成标签,学习与合成器无关的内容特征,并结合对抗学习抑制内容流中的合成器成分。最终分类基于合成器与内容特征拼接。为进一步提升对不同合成器特性的鲁棒性,提出合成器特征增强策略:随机混合真实与伪造音频特征风格,并随机打乱合成器与内容特征。该策略有效提升特征多样性,模拟更多特征组合。

原文摘要 · Abstract (English)

AI-synthesized speech, also known as deepfake speech, has recently raised significant concerns due to the rapid advancement of speech synthesis and speech conversion techniques. Previous works often rely on distinguishing synthesizer artifacts to identify deepfake speech. However, excessive reliance on these specific synthesizer artifacts may result in unsatisfactory performance when addressing speech signals created by unseen synthesizers. In this paper, we propose a robust deepfake speech detection method that employs feature decomposition to learn synthesizer-independent content features as complementary for detection. Specifically, we propose a dual-stream feature decomposition learning strategy that decomposes the learned speech representation using a synthesizer stream and a content stream. The synthesizer stream specializes in learning synthesizer features through supervised training with synthesizer labels. Meanwhile, the content stream focuses on learning synthesizer-independent content features, enabled by a pseudo-labeling-based supervised learning method. This method randomly transforms speech to generate speed and compression labels for training. Additionally, we employ an adversarial learning technique to reduce the synthesizer-related components in the content stream. The final classification is determined by concatenating the synthesizer and content features. To enhance the model's robustness to different synthesizer characteristics, we further propose a synthesizer feature augmentation strategy that randomly blends the characteristic styles within real and fake audio features and randomly shuffles the synthesizer features with the content features. This strategy effectively enhances the feature diversity and simulates more feature combinations.

语音伪造深度伪造检测特征分解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。