arXiv:2505.19119cs.SDcs.AI2025-05被引 2

用通用扰动保护语音隐私,防零样本克隆

CloneShield: A Framework for Universal Perturbation Against Zero-Shot Voice Cloning

  • 设计时间域对抗扰动,无需文本先验信息
  • 保护后音频质量接近原音(PESQ=3.90),克隆效果严重下降(PESQ=1.07)
  • 适合语音隐私保护场景,尤其对高危克隆攻击有效

近期文本到语音(TTS)语音克隆技术突破引发了严重隐私担忧,仅需几秒参考音频即可高精度复现说话人声音特征。本文提出CloneShield,一种针对零样本语音克隆的通用时域对抗扰动框架。该方法在无文本先验条件下,实现跨说话人与语句的鲁棒防护。通过多目标优化建模扰动生成,并引入多梯度下降算法(MGDA)确保对多样化语句的有效性。为保持听觉自然性,基于梅尔频谱表示分解并逐样本微调扰动。实验在三个先进零样本TTS系统、五个基准数据集及60名人类听者评估中显示:保护后音频质量接近原音(PESQ=3.90,SRS=0.93),而克隆样本的说话人相似性与语音质量显著下降(PESQ=1.07,SRS=0.08),证明其有效性与隐蔽性。

原文摘要 · Abstract (English)

Recent breakthroughs in text-to-speech (TTS) voice cloning have raised serious privacy concerns, allowing highly accurate vocal identity replication from just a few seconds of reference audio, while retaining the speaker's vocal authenticity. In this paper, we introduce CloneShield, a universal time-domain adversarial perturbation framework specifically designed to defend against zero-shot voice cloning. Our method provides protection that is robust across speakers and utterances, without requiring any prior knowledge of the synthesized text. We formulate perturbation generation as a multi-objective optimization problem, and propose Multi-Gradient Descent Algorithm (MGDA) to ensure the robust protection across diverse utterances. To preserve natural auditory perception for users, we decompose the adversarial perturbation via Mel-spectrogram representations and fine-tune it for each sample. This design ensures imperceptibility while maintaining strong degradation effects on zero-shot cloned outputs. Experiments on three state-of-the-art zero-shot TTS systems, five benchmark datasets and evaluations from 60 human listeners demonstrate that our method preserves near-original audio quality in protected inputs (PESQ = 3.90, SRS = 0.93) while substantially degrading both speaker similarity and speech quality in cloned samples (PESQ = 1.07, SRS = 0.08).

语音克隆对抗防御隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。