用说话人特有隐变量做水印，让语音克隆后仍可追踪
VoiceMark: Zero-Shot Voice Cloning-Resistant Watermarking Approach Leveraging Speaker-Specific Latents
- 用说话人特有隐变量承载水印，实现零样本克隆场景下的水印传递
- 在多个零样本克隆模型上检测准确率达95%以上，远超现有方法的约50%
- 适合需防语音伪造的版权保护、身份认证等应用场景
语音克隆（VC）抵抗型水印是用于追踪和防止未经授权克隆的新技术。现有方法通过在带水印音频上训练传统克隆模型来追踪，但在零样本克隆场景下失效——此时模型仅凭音频提示生成语音而无需训练。为此，我们提出首个零样本克隆抵抗型水印方法VoiceMark，利用说话人特有隐变量作为水印载体，使水印能穿过零样本克隆过程传递至合成语音中。此外，引入克隆模拟增强与基于语音活动检测（VAD）的损失函数，提升对失真的鲁棒性。在多个零样本克隆模型上的实验表明，VoiceMark在零样本克隆合成后水印检测准确率超过95%，显著优于现有方法的约50%。代码与演示见：https://huggingface.co/spaces/haiyunli/VoiceMark
原文摘要 · Abstract (English)
Voice cloning (VC)-resistant watermarking is an emerging technique for tracing and preventing unauthorized cloning. Existing methods effectively trace traditional VC models by training them on watermarked audio but fail in zero-shot VC scenarios, where models synthesize audio from an audio prompt without training. To address this, we propose VoiceMark, the first zero-shot VC-resistant watermarking method that leverages speaker-specific latents as the watermark carrier, allowing the watermark to transfer through the zero-shot VC process into the synthesized audio. Additionally, we introduce VC-simulated augmentations and VAD-based loss to enhance robustness against distortions. Experiments on multiple zero-shot VC models demonstrate that VoiceMark achieves over 95% accuracy in watermark detection after zero-shot VC synthesis, significantly outperforming existing methods, which only reach around 50%. See our code and demos at: https://huggingface.co/spaces/haiyunli/VoiceMark
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。