arXiv:2511.07099cs.SDcs.AI2025-11NeurIPS

为端到端语音合成设计对抗防御框架,防克隆诈骗

E2E-VGuard: Adversarial Prevention for Production LLM-based End-To-End Speech Synthesis

  • 用编码器集成+特征提取保护音色,对抗声纹篡改
  • 引入心理声学模型,扰动不可感知且有效防御攻击
  • 适配真实场景,支持开源与商用接口的防护

语音合成技术已广泛应用于日常生活,但语音克隆等恶意攻击带来严重安全风险。现有防御方法难以应对基于生产级大语言模型(LLM)的端到端(E2E)语音合成系统。以往研究依赖人工标注文本,而实际中越来越多采用自动语音识别(ASR)生成文本,如通过商业API实现语音克隆。为此,本文提出E2E-VGuard,一种主动防御框架,针对两大新威胁:(1)生产级LLM驱动的语音合成,(2)由ASR驱动的新型攻击。通过编码器集成配合特征提取器保护音色,同时利用面向ASR的对抗样本干扰发音;结合心理声学模型确保扰动不可察觉。在16个开源合成器和3个商用API上,覆盖中英文数据集进行评估,验证了其在音色与发音保护上的有效性,并完成真实部署验证。代码与演示页见https://wxzyd123.github.io/e2e-vguard/

原文摘要 · Abstract (English)

Recent advancements in speech synthesis technology have enriched our daily lives, with high-quality and human-like audio widely adopted across real-world applications. However, malicious exploitation like voice-cloning fraud poses severe security risks. Existing defense techniques struggle to address the production large language model (LLM)-based speech synthesis. While previous studies have considered the protection for fine-tuning synthesizers, they assume manually annotated transcripts. Given the labor intensity of manual annotation, end-to-end (E2E) systems leveraging automatic speech recognition (ASR) to generate transcripts are becoming increasingly prevalent, e.g., voice cloning via commercial APIs. Therefore, this E2E speech synthesis also requires new security mechanisms. To tackle these challenges, we propose E2E-VGuard, a proactive defense framework for two emerging threats: (1) production LLM-based speech synthesis, and (2) the novel attack arising from ASR-driven E2E scenarios. Specifically, we employ the encoder ensemble with a feature extractor to protect timbre, while ASR-targeted adversarial examples disrupt pronunciation. Moreover, we incorporate the psychoacoustic model to ensure perturbative imperceptibility. For a comprehensive evaluation, we test 16 open-source synthesizers and 3 commercial APIs across Chinese and English datasets, confirming E2E-VGuard's effectiveness in timbre and pronunciation protection. Real-world deployment validation is also conducted. Our code and demo page are available at https://wxzyd123.github.io/e2e-vguard/.

语音合成对抗防御大模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。