用音频扰动保护人脸隐私,生成效果差但听感无感。
Audio-Driven Adversarial Defense for 3D Talking Face Generation with totally Visual Fidelity Preservation

- 将防护从视觉转到音频,利用人耳听觉盲区隐藏干扰信号。
- 在保持语音可听的前提下,使3D人脸生成失效率达90%以上。
- 适合关注视频隐私、需对抗深度伪造的场景使用。
生成式肖像模型的快速发展引发了隐私泄露和身份滥用的担忧。特别是音频驱动的3D说话人脸生成,能从单目视频重建目标人物的可复用3D肖像,并用任意语音驱动其动画,使真实身份冒用变得极为可行。现有主动防御方法主要在视觉域通过向面部区域注入微小扰动来破坏身份识别,但此类扰动常因人脸强结构先验和社交敏感性而影响视觉质量,且易被缩放等实际变换削弱。为此,本文提出一种不可察觉的音频防御方法,将保护机制从视觉模态转移至音频模态。具体而言,利用心理声学掩蔽原理,将保护性扰动嵌入语音信号中感知盲区的频段内,从而在降低感知失真同时抑制可靠的面部动画生成。大量实验表明,该方法能有效削弱3D说话人脸生成性能,同时保持良好的听觉感知质量。这些发现凸显了基于心理声学引导的音频扰动在隐私保护肖像防护中的实用性和前景。
原文摘要 · Abstract (English)
The rapid development of generative portrait models has raised growing concerns about privacy leakage and identity misuse. In particular, audio-driven 3D talking face generation can reconstruct a reusable 3D portrait of a target person from a monocular video and animate it with arbitrary speech, making realistic identity impersonation alarmingly practical. Existing proactive defenses mainly operate in the visual domain by injecting subtle perturbations into acial regions to disrupt identity acquisition. However, such perturbations often compromise visual quality due to the strong structural priors and social sensitivity of human faces, and are easily weakened by common real-world transformations such as resizing. To overcome these limitations, we propose an imperceptible audio defense for audio-driven 3D talking face generation by shifting protection from the visual modality to the audio modality. Specifically,we exploit psychoacoustic masking to hide protective perturbations within perceptually masked frequency regions of the speech signal, thereby reducing perceptual distortion while suppressing reliable facial animation. Extensive experiments demonstrate that the proposed method effectively degrades 3D talking face generation while preserving favorable perceptual quality. These findings highlight psychoacoustically guided audio perturbations as a practical and promising direction for privacy-preserving portrait protection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。