构建两个新数据集，提升音乐源分离模型在真实演出环境下的表现。
CrowdioSet and PaRIRset: Two Datasets Towards Live Music Source Separation
- 用真实环境音与合成合唱音构建噪声数据集CrowdioSet。
- 引入40个场馆的立体声混响响应数据，提升分离效果。
- 适合研究真实场景音频分离或声学建模的研究者。
当前音乐源分离（MSS）模型在真实演出录音上泛化能力差，因其仅在录音室数据上训练，忽略场地声学、音响系统响应和观众噪声。为此，本文提出两个新数据集：第一，CrowdioSet，包含4800段来自Freesound的真实环境音轨及基于零样本语音转换生成的MUSDB18和MOISESDB中人声的合成合唱音；该数据集有效支持对现场录音的降噪，显著提升客观与主观分离效果。第二，PaRIRset，一个在40个专业演出场馆通过麦克风阵列采集的立体声混响响应（RIR）数据集。实验表明，使用PaRIRset中的RIR可使MSS模型性能优于仅使用语音增强任务中真实RIR的情况。所有数据、代码、模型权重均开源。
原文摘要 · Abstract (English)
Most Music Source Separation (MSS) models do not generalize well to live music recordings because they are trained on studio recordings alone, disregarding the venue acoustics, the speaker system's response and audience noise. We propose to bridge this gap by providing and training a model on two novel datasets. First, we present CrowdioSet: a noise dataset comprising 4800 real ambience tracks from Freesound and synthetic sing-alongs for the vocals in MUSDB18 and MOISESDB datasets, generated from zero-shot singing voice conversions. CrowdioSet enables effective audio denoising for live recordings, resulting in superior separation both in objective and subjective evaluations. Second, we introduce PaRIRset, a stereo impulse response dataset captured across 40 professional concert venues using a microphone array. Our results show that adding PaRIRset RIRs increases the performance of a MSS model compared to using real RIRs from Speech Enhancement tasks alone. We make the examples, code, model weights, PaRIRset, and CrowdioSet freely available to the public.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。