arXiv:2606.30791cs.SDeess.AS2026-06

通过探针筛选语音模型深层特征,提升音频伪造检测的跨域泛化能力。

Probing-Guided Layer Selection from Self-Supervised Speech Models for Generalizable Audio Deepfake Detection

论文配图:Probing-Guided Layer Selection from Self-Supervised Speech Models for Generalizable Audio Deepfake Detection
图 1 · 摘自论文原文
  • 用轻量级探针提前评估各层跨域判别力,选出关键深度区域
  • 仅融合选定层,参数量仅134万,错误率降低28%
  • 适配不同主干模型,避免固定层数选择,适合实际部署

音频深度伪造检测系统常因依赖特定攻击或录制条件而泛化能力差。自监督语音模型提供多层丰富表示,但现有方法要么仅用单层,要么盲目融合所有层,且层重要性需训练后才能揭示。本文提出一种模型无关、两阶段的方法,在任务模型训练前识别出有效深度区域。第一阶段,使用轻量级XGBoost探针评估各Transformer层的跨域判别能力,生成层排序;第二阶段,紧凑神经分类器通过每层注意力池化与共享瓶颈投影,仅融合选定层,主干保持冻结。在三个主干上应用,发现两个关键结论:其一,有效层集中于特定深度区而非单一最优位置,区域内替换误差在多种子噪声范围内,越界则性能下降最高达5倍;其二,探针结果具有主干特异性,非通用层选择。在XLS-R-300M上,仅使用4个探针选定层,134万可训练参数,在In-The-Wild测试集上达到4.94% ± 0.32%等错误率,跨域平均5.07%(四数据集),相比先前最佳冻结主干结果(Xiao and Vu, 2025)提升28%相对性能(使用全部25层,相同训练数据)。

原文摘要 · Abstract (English)

Audio deepfake detection systems often fail to generalize across domains because they rely on features tied to specific attacks or recording conditions. Self-supervised speech models offer rich multi-layer representations, yet existing approaches either use a single layer or fuse all layers indiscriminately, and only reveal layer importance after training. We propose a model-agnostic, two-stage methodology that identifies informative depth zones before any task-specific model is trained. In the first stage, lightweight XGBoost probes evaluate each transformer layer's cross-domain discriminative power, producing a layer ranking. In the second stage, a compact neural classifier fuses only the selected layers through per-layer attention pooling and a shared bottleneck projection, while the backbone remains frozen. Applied across three backbones, the probing reveals two key findings. First, informative layers cluster in depth zones rather than at uniquely optimal positions: within-zone substitutions fall within multi-seed noise, while zone violations degrade performance by up to 5x. Second, the probing produces backbone-specific selections rather than a fixed layer recipe. On XLS-R-300M, four probing-selected layers with 1.34M trainable parameters achieve 4.94 +/- 0.32% equal error rate on In-The-Wild and 5.07% cross-domain average over four shared datasets, a 28% relative improvement over the best prior frozen-backbone result (Xiao and Vu, 2025) using all 25 layers with identical training data.

音频伪造检测自监督学习层选择泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。