arXiv:2501.04942cs.SDeess.AS2025-01被引 1

用无监督图学习检测音频伪造,只需5%标注数据就达高精度。

SIGNL: A Label-Efficient Audio Deepfake Detection System via Spectral-Temporal Graph Non-Contrastive Learning

  • 构建频时双视角图模型,从声谱图提取音频结构特征。
  • 仅用5%标注数据,在ASVspoof2021上实现7.88%误报率。
  • 无需负样本的自监督预训练,适合数据稀缺场景。

随着合成语音愈发逼真且易获取,音频伪造检测变得至关重要。现有方法(如使用图神经网络建模频率与时间依赖性)虽具潜力,但需大量标注数据,限制了实际应用。标签高效替代方案如基于图的非对比学习可从无标签数据中学习有效表示,无需负样本。然而,当前图非对比方法仅适用于单视图图表示,无法直接用于具有独特频时结构的音频信号。为此,本文提出SIGNL(Spectral-temporal vIsion Graph Non-contrastive Learning),一种标签高效的音频伪造检测系统。SIGNL基于音频的视觉表示(如声谱图或时间-频率编码),将其转换为频域与时域图,进行结构化特征提取。通过图卷积编码器学习互补的频时特征,有效捕捉音频独特特性。编码器在增强图对上采用非对比自监督学习策略进行预训练,实现无需标签的有效表征学习。随后在少量标注数据上微调,用于下游伪造检测任务。SIGNL在多个音频伪造检测基准上表现优异,包括在ASVspoof 2021 DF上达到7.88% EER、在ASVspoof 5上达到3.95% EER(仅用5%标注数据)。其泛化能力亦强,在训练于CFAD的情况下,于In-The-Wild数据集上达到10.16% EER。

原文摘要 · Abstract (English)

Audio deepfake detection is increasingly important as synthetic speech becomes more realistic and accessible. Recent methods, including those using graph neural networks (GNNs) to model frequency and temporal dependencies, show strong potential but need large amounts of labeled data, which limits their practical use. Label-efficient alternatives like graph-based non-contrastive learning offer a potential solution, as they can learn useful representations from unlabeled data without using negative samples. However, current graph non-contrastive approaches are built for single-view graph representations and cannot be directly used for audio, which has unique spectral and temporal structures. Bridging this gap requires dual-view graph modeling suited to audio signals. In this work, we introduce SIGNL (Spectral-temporal vIsion Graph Non-contrastive Learning), a label-efficient expert system for detecting audio deepfakes. SIGNL operates on the visual representation of audio, such as spectrograms or other time-frequency encodings, transforming them into spectral and temporal graphs for structured feature extraction. It then employs graph convolutional encoders to learn complementary frequency-time features, effectively capturing the unique characteristics of audio. These encoders are pre-trained using a non-contrastive self-supervised learning strategy on augmented graph pairs, enabling effective representation learning without labeled data. The resulting encoders are then fine-tuned on minimal labelled data for downstream deepfake detection. SIGNL achieves strong performance on multiple audio deepfake detection benchmarks, including 7.88% EER on ASVspoof 2021 DF and 3.95% EER on ASVspoof 5 using only 5% labeled data. It also generalizes well to unseen conditions, reaching 10.16% EER on the In-The-Wild dataset when trained on CFAD.

音频伪造图神经网络自监督学习低标签

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。