用真实环境噪声测试音频指纹,发现现有模型性能大幅下降。
Contrastive and Transfer Learning for Effective Audio Fingerprinting through a Real-World Evaluation Protocol
- 设计新评估协议,模拟手机录音的渐进降噪场景
- 引入滤波增强训练,使模型在强噪声下仍保持56.5%检出率
- 基于变换器的模型跨域迁移,10秒查询准确率达97%
近期歌曲识别方法利用深度神经网络从原始波形中直接学习紧凑的音频指纹。尽管在受控条件下表现良好,但在真实场景中(如移动设备麦克风在嘈杂环境中录制)准确率显著下降。本文提出一种新型评估协议,生成同一音频的三段录音,噪声水平逐级升高。结果表明,两种先进CNN模型在此协议下的性能远低于以往报告的基准。我们强调了训练时数据增强管道的重要性:通过在增强中引入低通和高通滤波器,显著提升了两个系统的性能。此外,我们构建了一个带有定制投影模块的变压器模型,并证明从语义相关领域迁移知识可获得更鲁棒的解决方案。该变压器架构在所有噪声水平和查询时长下均优于CNN模型;在低噪声下,1秒查询达到47.99%,10秒查询达97%,分别超越次优模型14%和18.5%。在重度噪声下,15秒查询检测率达到56.5%。所有实验基于超过10万首歌曲的公开大规模数据集,查询匹配5600万向量数据库。
原文摘要 · Abstract (English)
Recent advances in song identification leverage deep neural networks to learn compact audio fingerprints directly from raw waveforms. While these methods perform well under controlled conditions, their accuracy drops significantly in real-world scenarios where the audio is captured via mobile devices in noisy environments. In this paper, we introduce a novel evaluation protocol designed to better reflect such real-world conditions. We generate three recordings of the same audio, each with increasing levels of noise, captured using a mobile device's microphone. Our results reveal a substantial performance drop for two state-of-the-art CNN-based models under this protocol, compared to previously reported benchmarks. Additionally, we highlight the critical role of the augmentation pipeline during training with contrastive loss. By introduction low pass and high pass filters in the augmentation pipeline we significantly increase the performance of both systems in our proposed evaluation. Furthermore, we develop a transformer-based model with a tailored projection module and demonstrate that transferring knowledge from a semantically relevant domain yields a more robust solution. The transformer architecture outperforms CNN-based models across all noise levels, and query durations. In low noise conditions it achieves 47.99% for 1-sec queries, and 97% for 10-sec queries in finding the correct song, surpassing by 14%, and by 18.5% the second-best performing model, respectively, Under heavy noise levels, we achieve a detection rate 56.5% for 15-second query duration. All experiments are conducted on public large-scale dataset of over 100K songs, with queries matched against a database of 56 million vectors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。