通过分离高低频特征,提升深度伪造音频检测的泛化能力。
SONAR: Spectral-Contrastive Audio Residuals for Generalizable Deepfake Detection
- 将音频分解为低频主干与高频残差两部分,分别处理
- 在ASVspoof 2021上达到当前最佳性能,收敛速度提升4倍
- 适合需要依赖微弱高频线索的伪造检测任务
深度伪造音频检测器在分布外输入上仍表现不佳,核心原因是频谱偏差:神经网络优先学习低频结构,导致伪造生成器留下高频伪影,而现有检测器又未能有效利用这些伪影。为此,本文提出频谱对比音频残差框架SONAR,通过XLSR编码器捕获主导低频内容,同时在克隆路径前加入可学习的高通滤波器(SRM),提取微弱高频残差。频率交叉注意力融合长短期频域依赖关系,频率感知的Jensen-Shannon对比损失将真实内容-噪声对拉近,推远伪造嵌入,加速优化并增强决策边界。在ASVspoof 2021和野外基准测试中,SONAR达到顶尖性能,收敛速度比强基线快四倍。该方法将潜在空间划分为自然高频与失真高频两个互斥流形,实现数据驱动、频率引导的对比学习。由于仅作用于表示层,SONAR具备架构无关性,未来可无缝集成至任意模型或模态。
原文摘要 · Abstract (English)
Deepfake (DF) audio detectors still struggle to generalize to out of distribution inputs. A central reason is spectral bias, the tendency of neural networks to learn low-frequency structure before high-frequency (HF) details, which both causes DF generators to leave HF artifacts and leaves those same artifacts under-exploited by common detectors. To address this gap, we propose Spectral-cONtrastive Audio Residuals (SONAR), a frequency-guided framework that explicitly disentangles an audio signal into complementary representations. An XLSR encoder captures the dominant low-frequency content, while the same cloned path, preceded by learnable SRM, value-constrained high-pass filters, distills faint HF residuals. Frequency cross-attention reunites the two views for long- and short-range frequency dependencies, and a frequency-aware Jensen-Shannon contrastive loss pulls real content-noise pairs together while pushing fake embeddings apart, accelerating optimization and sharpening decision boundaries. Evaluated on the ASVspoof 2021 and in-the-wild benchmarks, SONAR attains state-of-the-art performance and converges four times faster than strong baselines. By elevating faint high-frequency residuals to first-class learning signals, SONAR unveils a fully data-driven, frequency-guided contrastive framework that splits the latent space into two disjoint manifolds: natural-HF for genuine audio and distorted-HF for synthetic audio, thereby sharpening decision boundaries. Because the scheme operates purely at the representation level, it is architecture-agnostic and, in future work, can be seamlessly integrated into any model or modality where subtle high-frequency cues are decisive.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。