SegTune让音乐生成可分段精细控制，歌词与旋律对齐更精准。
SegTune: Structured and Fine-Grained Control for Song Generation
- 通过分段提示词在时间窗内注入音乐描述，实现局部控制。
- 使用大模型预测歌词时间戳，提升段落时长与对齐精度。
- 适合需要精准结构控制的音乐创作与自动化作曲研究者。
近期歌曲生成研究在基于歌词或全局文本提示生成歌曲方面取得了显著进展。然而，大多数现有系统无法建模歌曲中随时间变化的属性，限制了对音乐结构与动态的细粒度控制。本文提出SegTune，一种非自回归的结构化可控歌曲生成框架。SegTune通过将局部音乐描述与歌曲段落对齐，实现分段级控制；分段提示词通过时间广播注入对应时间窗口，而全局提示则影响全曲以保证风格一致性。为获得精确的段落时长并实现歌词与音乐的精准对齐，我们引入一个基于大模型的时长预测器，用于自回归生成带时间戳的歌词（LRC格式）。我们还构建了一个大规模高质量数据管道，收集带有对齐歌词与提示的歌曲，并提出新的评估指标以衡量分段对齐与人声属性一致性。实验结果表明，SegTune在可控性与音乐连贯性上均优于现有基线方法。
原文摘要 · Abstract (English)
Recent advancements in song generation have shown promising results in generating songs from lyrics and/or global text prompts. However, most existing systems lack the ability to model the temporally varying attributes of songs, limiting fine-grained control over musical structure and dynamics. In this paper, we propose SegTune, a non-autoregressive framework for structured and controllable song generation. SegTune enables segment-level control by allowing users or large language models to specify local musical descriptions aligned to song sections.The segmental prompts are injected into the model by temporally broadcasting them to corresponding time windows, while global prompts influence the whole song to ensure stylistic coherence. To obtain accurate segment durations and enable precise lyric-to-music alignment, we introduce an LLM-based duration predictor that autoregressively generates sentence-level timestamped lyrics in LRC format. We further construct a large-scale data pipeline for collecting high-quality songs with aligned lyrics and prompts, and propose new evaluation metrics to assess segment-level alignment and vocal attribute consistency. Experimental results show that SegTune achieves superior controllability and musical coherence compared to existing baselines. See https://cai525.github.io/SegTune_demo for demos of our work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。