通过融合语音与环境音的自监督表示，提升深度伪造音频检测精度
Deepfake Audio Detection Using Self-supervised Fusion Representations
- 双分支架构分别提取语音与环境音上下文特征
- 测试集上环境音伪造检测EER达16.54%，F1-score为70.20%
- 适合关注多模态音频安全与伪造检测的研究者
本文针对环境感知语音与声音深度伪造检测挑战赛（ESDD2 2026）提出一种组件级深度伪造检测方案，基于CompSpoofV2数据集，该数据集允许语音与环境音独立被篡改。为此，提出一种双分支深度伪造检测框架，联合建模输入音频中的语音与环境上下文表征。采用XLS-R模型提取语音特征，BEATs模型提取环境音特征，获得互补的上下文表示。引入匹配头（Matching Head），通过统计归一化与表示交互建模差异，以估计原始类别。同时，利用多头交叉注意力实现语音与环境成分间的有效信息交换。经过残差连接与层归一化优化后的表示送入AASIST分类器，输出语音与环境层面的欺骗概率预测。系统在测试集上取得70.20%的F1-score和16.54%的环境音等错误率（EER），优于基线系统。
原文摘要 · Abstract (English)
This paper describes a submission to the Environment-Aware Speech and Sound Deepfake Detection Challenge (ESDD2) 2026, which addresses component-level deepfake detection using the CompSpoofV2 dataset, where speech and environmental sounds may be independently manipulated. To address this challenge, a dual-branch deepfake detection framework is proposed to jointly model speech and environmental contextual representations from input audio. Two pretrained models, XLS-R for speech and BEATs for environmental sound, are used to extract complementary contextual representations. A Matching Head is introduced to model representation differences through statistical normalization and representation interaction, enabling estimation of the original class. In parallel, multi-head cross-attention enables effective information exchange between speech and environmental components. The refined representations are processed with residual connections and layer normalization, and passed to an AASIST classifier to predict speech-based and environment-based spoofing probabilities. The model outputs original, speech, and environment predictions. On the test set, the proposed system achieves an F1-score of 70.20% and an environmental EER of 16.54%, outperforming the baseline system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。