轻量级模型SwiftF0实现高精度实时音高检测,适合低资源设备部署。
SwiftF0: Fast and Accurate Monophonic Pitch Detection
- 基于合成语音数据训练,提升噪声环境下的音高估计鲁棒性。
- 在10dB信噪比下达到91.8%准确率,比基线快42倍且参数仅9.6万。
- 提供精确音高真值数据集SpeechSynth与开源评测框架,助力研究验证。
在噪声环境下,尤其在资源受限设备上实现高精度、实时的单音调音高估计仍是音频处理中的开放挑战。本文提出轻量级神经模型SwiftF0,成为单音调音高估计的新基准。通过在多样化的语音、音乐及合成数据集上进行大规模数据增强训练,SwiftF0实现了跨声学领域的强泛化能力,同时保持计算高效。其在10 dB信噪比下达到91.80%的调和均值(HM),较基线CREPE提升超过12个百分点,仅比纯净音频下降2.3分。模型仅需95,842个参数,在CPU上运行速度比CREPE快约42倍,非常适合高效实时部署。为解决语音语料中缺乏精确音高真值的问题(传统依赖算法估计或喉镜信号),我们构建了合成语音数据集SpeechSynth,由音素级语音合成模型生成,可提供精确、按需获取的音高曲线,支持更可靠的模型训练与评估。此外,提出统一指标融合六项互补性能度量,构建开源音高评测套件。项目演示地址:https://swift-f0.github.io/,源代码:https://github.com/lars76/swift-f0,评测框架:https://github.com/lars76/pitch-benchmark。
原文摘要 · Abstract (English)
Accurate and real-time monophonic pitch estimation in noisy conditions, particularly on resource-constrained devices, remains an open challenge in audio processing. We present \emph{SwiftF0}, a novel, lightweight neural model that sets a new state-of-the-art for monophonic pitch estimation. Through training on diverse speech, music, and synthetic datasets with extensive data augmentation, SwiftF0 achieves robust generalization across acoustic domains while maintaining computational efficiency. SwiftF0 achieves a 91.80\% harmonic mean (HM) at 10 dB SNR, outperforming baselines like CREPE by over 12 percentage points and degrading by only 2.3 points from clean audio. SwiftF0 requires only 95,842 parameters and runs approximately 42x faster than CREPE on CPU, making it ideal for efficient, real-time deployment. To address the critical lack of perfectly accurate ground truth pitch in speech corpora (which typically rely on algorithmic estimators or laryngograph signals), we introduce \emph{SpeechSynth}. This synthetic speech dataset, generated by a phoneme-level TTS model, provides exact, on-demand ground-truth pitch curves, enabling more robust model training and evaluation. Furthermore, we propose a unified metric, combining six complementary performance measures for comprehensive and reliable pitch evaluation, and release an open-source pitch benchmark suite. A live demo of SwiftF0 is available at https://swift-f0.github.io/, the source code at https://github.com/lars76/swift-f0, and the benchmark framework at https://github.com/lars76/pitch-benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。