REFFLY可编辑歌词使其匹配旋律，支持翻译与风格迁移。
REFFLY: Melody-Constrained Lyrics Editing Model
- 用合成数据训练修订模块，将普通文本转为合拍歌词。
- 在音乐性与文本质量上比基线模型提升25%。
- 适合需要改词适配旋律的创作、翻译或风格转换场景。
自动旋律到歌词（M2L）生成旨在创建与给定旋律相匹配的歌词。以往方法多从零生成，而歌词修订（将普通文本修改为符合旋律的歌词）更具灵活性和实用性，适用于关键词/主题输入生成、保留旋律的歌曲翻译，以及风格迁移。本文提出首个歌词修订框架REFFLY（REvision Framework For LYrics），通过自建的合成旋律对齐歌词数据集训练歌词修订模块，实现将普通文本转化为旋律契合的歌词。为进一步提升修订能力，设计无需训练的启发式策略，兼顾语义一致性与音乐连贯性。实验表明，REFFLY在多种任务中表现优异，相比强基线模型Lyra（Tian et al., 2023）和GPT-4，在音乐性与文本质量上均提升25%。
原文摘要 · Abstract (English)
Automatic melody-to-lyric (M2L) generation aims to create lyrics that align with a given melody. While most previous approaches generate lyrics from scratch, revision, editing plain text draft to fit it into the melody, offers a much more flexible and practical alternative. This enables broad applications, such as generating lyrics from flexible inputs (keywords, themes, or full text that needs refining to be singable), song translation (preserving meaning across languages while keeping the melody intact), or style transfer (adapting lyrics to different genres). This paper introduces REFFLY (REvision Framework For LYrics), the first revision framework for editing and generating melody-aligned lyrics. We train the lyric revision module using our curated synthesized melody-aligned lyrics dataset, enabling it to transform plain text into lyrics that align with a given melody. To further enhance the revision ability, we propose training-free heuristics aimed at preserving both semantic meaning and musical consistency throughout the editing process. Experimental results demonstrate the effectiveness of REFFLY across various tasks (e.g. lyrics generation, song translation), showing that our model outperforms strong baselines, including Lyra (Tian et al., 2023) and GPT-4, by 25% in both musicality and text quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。