用线性RNN生成长达一分钟的音乐波形，效果超越现有方法。
Linear RNNs for autoregressive generation of long music samples
- 采用线性RNN结构，结合上下文并行训练长序列。
- 在小数据集上实现最优对数似然与感知评分。
- 适合需要高质量长音频生成的研究者或创作者。
直接以自回归方式学习生成原始音频波形是一项挑战，因原始序列过长且存在多时间尺度的重要结构。传统基于循环神经网络、因果卷积和自注意力的方法在此任务上成效有限。然而，近期研究表明深度状态空间模型（又称线性RNN）在此场景下具有高效率。本文进一步推进线性RNN在原始音频建模中的应用，研究不同架构选择的影响，并利用上下文并行性实现长达一分钟（100万标记）序列的训练。我们提出HarmonicRNN模型，在小规模数据集上达到当前最优的对数似然与感知指标。
原文摘要 · Abstract (English)
Directly learning to generate audio waveforms in an autoregressive manner is a challenging task, due to the length of the raw sequences and the existence of important structure on many different timescales. Traditional approaches based on recurrent neural networks, as well as causal convolutions and self-attention, have only had limited success on this task. However, recent work has shown that deep state space models, also referred to as linear RNNs, can be highly efficient in this context. In this work, we push the boundaries of linear RNNs applied to raw audio modeling, investigating the effects of different architectural choices and using context-parallelism to enable training on sequences up to one minute (1M tokens) in length. We present a model, HarmonicRNN, which attains state of the art log-likelihoods and perceptual metrics on small-scale datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。