通过自编码器提取伪造音频特征,实现高精度语音深度伪造溯源与模型识别。
Towards Reliable Audio Deepfake Attribution and Model Recognition: A Multi-Level Autoencoder-Based Framework
- 用仅训练于伪造音频的卷积自编码器提取注意力增强的隐层特征
- 在多个数据集上实现超95%的归属识别准确率,特定模型识别达96.31%
- 支持开放集场景,适合安全审计、反欺诈等可信语音应用
语音深度伪造的泛滥正威胁数字通信的信任基础。尽管检测技术已进步,但将伪造音频归因于其生成模型仍是未充分探索却至关重要的挑战。本文提出LAVA(语音溯源分层架构),一种基于卷积自编码器的层次化框架,该自编码器仅在伪造音频上训练,提取注意力增强的潜在表示。两个专用分类器分别执行:音频深度伪造归因(ADA)以识别生成技术,音频深度伪造模型识别(ADMR)以识别具体生成模型实例。为提升开放集条件下的鲁棒性,引入基于置信度的拒绝阈值。在ASVspoof2021、FakeOrReal和CodecFake上的实验显示,ADA分类器在所有数据集上F1分数均超过95%,ADMR模块在六类中达到96.31%的宏平均F1。对ASVpoof2019 LA中未见攻击的额外测试及误差传播分析进一步验证了LAVA的鲁棒性与可靠性。该框架首次在开放集条件下实现监督式深度伪造归因与模型识别,经公开基准验证,并公开发布模型与代码。
原文摘要 · Abstract (English)
The proliferation of audio deepfakes poses a growing threat to trust in digital communications. While detection methods have advanced, attributing audio deepfakes to their source models remains an underexplored yet crucial challenge. In this paper we introduce LAVA (Layered Architecture for Voice Attribution), a hierarchical framework for audio deepfake detection and model recognition that leverages attention-enhanced latent representations extracted by a convolutional autoencoder trained solely on fake audio. Two specialized classifiers operate on these features: Audio Deepfake Attribution (ADA), which identifies the generation technology, and Audio Deepfake Model Recognition (ADMR), which recognize the specific generative model instance. To improve robustness under open-set conditions, we incorporate confidence-based rejection thresholds. Experiments on ASVspoof2021, FakeOrReal, and CodecFake show strong performance: the ADA classifier achieves F1-scores over 95% across all datasets, and the ADMR module reaches 96.31% macro F1 across six classes. Additional tests on unseen attacks from ASVpoof2019 LA and error propagation analysis confirm LAVA's robustness and reliability. The framework advances the field by introducing a supervised approach to deepfake attribution and model recognition under open-set conditions, validated on public benchmarks and accompanied by publicly released models and code. Models and code are available at https://www.github.com/adipiz99/lava-framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。