用频域控制生成音乐，让模型像调音台一样调节结构。
Latent Fourier Transform
- 在隐空间做傅里叶变换，按时间尺度分离音乐模式。
- 可独立操控不同频段，生成保持指定时序特征的新曲子。
- 适合想精细控制音乐结构的创作者与研究者。
我们提出隐空间傅里叶变换（LatentFT），一种为生成式音乐模型提供新颖频域控制的框架。该方法结合扩散自编码器与隐空间傅里叶变换，按时间尺度分离音乐模式。训练时通过掩码隐变量的频域成分，获得可在推理阶段协同操控的表示。由此可基于参考样本生成变体与混音，同时保留目标时序尺度的特征，这些特征由隐空间频率指定。LatentFT 类似于音乐制作中的均衡器：传统均衡器作用于可听频率以塑造音色，而 LatentFT 作用于隐空间频率以塑造音乐结构。实验与听觉测试表明，相比基线方法，LatentFT 提升了条件遵循度与生成质量。我们还提出一种将隐空间频率单独听出的技术，并展示不同音乐属性分布在隐空间频谱的不同区域。结果表明，隐空间的频域控制提供了直观、连续的条件与混合轴，推动生成音乐模型向更可解释、更交互的方向发展。
原文摘要 · Abstract (English)
We introduce the Latent Fourier Transform (LatentFT), a framework that provides novel frequency-domain controls for generative music models. LatentFT combines a diffusion autoencoder with a latent-space Fourier transform to separate musical patterns by timescale. By masking latents in the frequency domain during training, our method yields representations that can be manipulated coherently at inference. This allows us to generate musical variations and blends from reference examples while preserving characteristics at desired timescales, which are specified as frequencies in the latent space. LatentFT parallels the role of the equalizer in music production: while traditional equalizers operates on audible frequencies to shape timbre, LatentFT operates on latent-space frequencies to shape musical structure. Experiments and listening tests show that LatentFT improves condition adherence and quality compared to baselines. We also present a technique for hearing frequencies in the latent space in isolation, and show different musical attributes reside in different regions of the latent spectrum. Our results show how frequency-domain control in latent space provides an intuitive, continuous frequency axis for conditioning and blending, advancing us toward more interpretable and interactive generative music models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。