用歌词对齐音频嵌入,高效实现音乐翻唱识别
Scalable Music Cover Retrieval Using Lyrics-Aligned Audio Embeddings
- 利用先进语音转录与文本嵌入模型训练,提升识别精度
- 推理时无需转录,计算量大幅降低,速度更快
- 适合需要快速部署的音乐版权管理场景
音乐翻唱识别(Music Cover Retrieval)旨在识别同一首歌曲的不同演绎版本,是曲库管理、版权保护和音乐检索的核心任务。当前主流方法多依赖和声与旋律特征,采用复杂的音频处理流程以应对翻唱间显著变化的音乐属性,但这类方法训练耗时长、计算开销大。相比之下,歌词在不同翻唱中具有高度一致性,但以往因难以从混响音频中准确提取而应用受限。早期方法框架简单,性能不足;近期系统虽表现更优,却依赖大型多模态模型,结构复杂。本文提出LIVI(Lyrics-Informed Version Identification),在保证识别精度与或优于传统和声方法的同时,通过训练阶段借助先进转录与文本嵌入模型进行监督,推理时完全去除转录步骤,显著降低计算负担,打破复杂流水线主导局面。
原文摘要 · Abstract (English)
Music Cover Retrieval, also known as Version Identification, aims to recognize distinct renditions of the same underlying musical work, a task central to catalog management, copyright enforcement, and music retrieval. State-of-the-art approaches have largely focused on harmonic and melodic features, employing increasingly complex audio pipelines designed to be invariant to musical attributes that often vary widely across covers. While effective, these methods demand substantial training time and computational resources. By contrast, lyrics constitute a strong invariant across covers, though their use has been limited by the difficulty of extracting them accurately and efficiently from polyphonic audio. Early methods relied on simple frameworks that limited downstream performance, while more recent systems deliver stronger results but require large models integrated within complex multimodal architectures. We introduce LIVI (Lyrics-Informed Version Identification), an approach that seeks to balance retrieval accuracy with computational efficiency. First, LIVI leverages supervision from state-of-the-art transcription and text embedding models during training to achieve retrieval accuracy on par with--or superior to--harmonic-based systems. Second, LIVI remains lightweight and efficient by removing the transcription step at inference, challenging the dominance of complexity-heavy pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。