让歌声歌词改写不走调、不变声,还更自然。
CLASVS: Continuous-Latent Autoregression for Melody-Preserving Lyric Editing in Singing Voice Synthesis

- 用连续潜变量自回归实现渐进式歌词编辑
- 中文双数据集上字错误率降低46.2%
- 适合需要精准歌词修改的语音合成场景
参考条件下的旋律保持型歌词编辑在替换词语的同时保留表演节奏、歌手身份与自然度。连续潜变量自回归避免了离散码本,支持可学习停止的逐步生成。编辑带来普通重建中不存在的矛盾:训练时参考信号与原歌词配对,推理时却要求新歌词覆盖原有歌词相关的参考信号;一个跟随源歌词的片段可能沿自回归历史传播。本文提出CLASVS,其状态控制转移(SCT)路由机制使目标歌词与参考旋律控制持续有效,向因果规划器反馈语义进展,并将前一潜变量块限制在局部过渡区。渐进状态控制锚定(PSCG)通过成对无编辑、内容一致的中文重建学习该约束。在两个中文基准上,CLASVS在四项操作中均优于离散自回归模型Vevo2,宏字错误率降低46.2%,同时保持旋律一致性、歌手相似性与感知质量。这些结果确立了无需乐谱标注的歌词编辑中强健的连续自回归工作点,并为更广泛的分步控制奠定基础。音频演示见项目页:https://piedpiperg.github.io/clasvs-demo/。
原文摘要 · Abstract (English)
Reference-conditioned melody-preserving lyric editing replaces words while retaining a performance's timing, singer identity, and naturalness. Continuous-latent autoregression avoids finite codebooks and offers stepwise generation with learned stopping. Editing creates a conflict absent from ordinary reconstruction: training pairs reference cues with original lyrics, whereas inference asks revised lyrics to override source-lyric-correlated cues; one source-following patch can propagate through AR history. We introduce CLASVS. Its State-Control-Transition (SCT) routing keeps target-lyric and reference-melody controls persistent, returns semantic feedback on phonetic progress to the causal planner, and confines the previous latent patch to the local Transition. Progressive State-Control Grounding (PSCG) learns this contract through paired-edit-free, content-consistent Mandarin reconstruction. On two Mandarin benchmarks, CLASVS improves all four operations over discrete-AR Vevo2 and reduces macro-PER by 46.2%, while maintaining melody, singer similarity, and perceptual quality. Together, these results establish a strong continuous-AR operating point for score-annotation-free lyric edits and a basis for broader stepwise control. Audio demonstrations are available on our project page: https://piedpiperg.github.io/clasvs-demo/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。