通过对比对齐损失，实现旋律与歌词的精准匹配。
Melody-Lyrics Matching with Contrastive Alignment Loss
- 采用自监督框架，利用旋律与歌词间的节奏、韵律等隐含关系。
- 无需标注对齐信息，可从海量配对歌曲中学习表示。
- 提出音节级歌词表示sylphone，提升匹配自然度与可唱性。
音乐与歌词的关联远超语义层面，如节奏与押韵、音符时长与音节重音、结构对应等概念性对应关系，在音乐信息检索领域仍鲜有研究。本文提出旋律-歌词匹配（Melody-Lyrics Matching, MLM）任务，即从文本源中为给定符号化旋律检索可能的歌词。不同于从零生成歌词，MLM着重挖掘旋律与歌词间的内在关联。我们提出一种基于对比对齐损失的自监督表征学习框架，能充分利用大量已有的旋律-歌词配对数据，且无需对齐标注。此外，我们引入sylphone——一种基于音素身份和元音重音激活的音节级歌词表示。实验结果与直观示例表明，该方法可匹配出连贯且可演唱的歌词。代码与匹配样例已开源：https://github.com/changhongw/mlm。
原文摘要 · Abstract (English)
The connection between music and lyrics is far beyond semantic bonds. Conceptual pairs in the two modalities such as rhythm and rhyme, note duration and syllabic stress, and structure correspondence, raise a compelling yet seldom-explored direction in the field of music information retrieval. In this paper, we present melody-lyrics matching (MLM), a new task which retrieves potential lyrics for a given symbolic melody from text sources. Rather than generating lyrics from scratch, MLM essentially exploits the relationships between melody and lyrics. We propose a self-supervised representation learning framework with contrastive alignment loss for melody and lyrics. This has the potential to leverage the abundance of existing songs with paired melody and lyrics. No alignment annotations are required. Additionally, we introduce sylphone, a novel representation for lyrics at syllable-level activated by phoneme identity and vowel stress. We demonstrate that our method can match melody with coherent and singable lyrics with empirical results and intuitive examples. We open source code and provide matching examples on the companion webpage: https://github.com/changhongw/mlm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。