首次融合图学习与开放集检测,精准追踪合成语音来源。
Bridging Attribution and Open-Set Detection using Graph-Augmented Instance Learning in Synthetic Speech
- 构建查询相关的图结构,用GNN分析语音生成器间关系。
- 结合KNN与置信度阈值,识别未知生成器的合成语音。
- 在DiffSSD和SingFake上表现优异,适合语音溯源与安全检测。
我们提出SIGNAL框架,统一解决合成语音溯源与开放集检测问题。该框架融合语音基础模型(SFMs)、图神经网络(GNN)与基于置信度的开放集分类。通过构建以查询为条件的生成器原型图,使GNN能够推理候选生成器间的关联;同时利用k-NN分支实现开放集检测。我们在DiffSSD数据集上评估,该数据集包含开源与商用扩散型语音合成系统生成的多样化真实与合成语音;并进一步在SingFake基准上测试泛化能力。结果表明,SIGNAL在两项任务中均持续提升性能,其中基于Mamba的嵌入表现尤为突出。据我们所知,这是首个将图学习与开放集检测结合用于合成语音溯源的研究。
原文摘要 · Abstract (English)
We propose a unified framework for not only attributing synthetic speech to its source but also for detecting speech generated by synthesizers that were not encountered during training. This requires methods that move beyond simple detection to support both detailed forensic analysis and open-set generalization. To address this, we introduce SIGNAL, a hybrid framework that combines speech foundation models (SFMs) with graph-based modeling and open-set-aware inference. Our framework integrates Graph Neural Networks (GNNs) and a k-Nearest Neighbor (KNN) classifier, allowing it to capture meaningful relationships between utterances and recognize speech that doesn`t belong to any known generator. It constructs a query-conditioned graph over generator class prototypes, enabling the GNN to reason over relationships among candidate generators, while the KNN branch supports open-set detection via confidence-based thresholding. We evaluate SIGNAL using the DiffSSD dataset, which offers a diverse mix of real speech and synthetic audio from both open-source and commercial diffusion-based TTS systems. To further assess generalization, we also test on the SingFake benchmark. Our results show that SIGNAL consistently improves performance across both tasks, with Mamba-based embeddings delivering especially strong results. To the best of our knowledge, this is the first study to unify graph-based learning and open-set detection for tracing synthetic speech back to its origin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。