让歌声改词不跑调也不变长，还能精准控制每句时长。
MeloDISinger: Melody-Aware & Duration-Preserving Singing Voice Editing with Audio Infilling
- 用跨注意力融合发音与旋律信息，实现时长精准分配。
- 通过流匹配音频修复，合成新歌词区域并保持原音上下文。
- 支持真实场景评估，适合音乐创作与语音编辑研究者。
文本驱动的歌唱语音编辑（SVE）旨在修改歌词的同时保留原始旋律、总时长和未编辑区域。本文提出 MeloDISinger，一种基于流匹配的 SVE 模型，实现旋律感知与时长保持的编辑。其核心模块 MeloDRP 预测固定预算下的时长比例，实现逐段时长控制。为实现旋律感知的时长分配，MeloDRP 通过交叉注意力融合发音线索与伪 MIDI 旋律上下文，同时采用时间重叠监督机制，促进音素与音符间的软对应关系。此外，我们使用流匹配声码器进行音频补全，以合成编辑区域并保留周围上下文。为进一步构建可行评估场景，我们引入基于 WhisperX 与大语言模型的时长感知歌词生成流程。实验表明，该方法在客观与主观评价中均达到当前最优性能。
原文摘要 · Abstract (English)
Text-based singing voice editing (SVE) aims to revise sung lyrics while preserving the original melody, total duration, and non-edited regions. In this paper, we propose MeloDISinger, a flow-matching-based SVE model for melody-aware and duration-preserving editing. Its core module, MeloDRP, predicts fixed-budget duration ratios, enabling explicit span-wise duration control. For melody-aware duration allocation, MeloDRP fuses phonetic cues with pseudo-MIDI melodic context through cross-attention, while temporal-overlap supervision encourages soft phoneme--note correspondences. We further use a flow-matching mel decoder for audio infilling to synthesize edited regions while preserving surrounding context. In addition, we introduce a duration-aware edited-lyric generation pipeline using WhisperX and an LLM to construct feasible evaluation scenarios. Experiments demonstrate state-of-the-art performance in both objective and subjective evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。