用Mamba替换注意力机制，让音乐超分辨率更快更省显存。
AEROMamba: An efficient architecture for audio super-resolution using generative adversarial networks and state space models
- 用状态空间模型Mamba替代Attention和LSTM，兼顾长程依赖与效率
- 在MUSDB上得分达66.74，比AERO提升11.2%，推理速度提升14倍
- 适合追求高效音频重建的开发者和音频工程师使用
音频超分辨率旨在通过生成高频内容来提升低分辨率信号质量。本文针对音乐超分辨率任务，改进了当前最先进的AERO架构：将所有网络层中的Attention与LSTM模块替换为Mamba（一种状态空间模型）。Mamba兼具注意力机制的建模能力与循环网络的递归特性，可有效替代原有模块。实验表明，该方法训练时所需GPU内存减少2-4倍，得益于其卷积式设计和对GPU内存层次结构的利用。推理过程中，由于递归特性，内存消耗保持恒定，避免了注意力机制导致的内存增长。在相同硬件下，推理速度提升14倍，显存占用降低5倍。主观听感测试（0-100分制）显示，AEROMamba优于AERO。在MUSDB数据集上，原始信号得分为38.22，AERO为60.03，而AEROMamba达到66.74；在PianoEval数据集上，三者得分分别为72.92、76.89和84.41。
原文摘要 · Abstract (English)
Audio super-resolution aims to enhance low-resolution signals by creating high-frequency content. In this work, we modify the architecture of AERO (a state-of-the-art system for this task) for music super-resolution. SPecifically, we replace its original Attention and LSTM layers with Mamba, a State Space Model (SSM), across all network layers. Mamba is capable of effectively substituting the mentioned modules, as it offers a mechanism similar to that of Attention while also functioning as a recurrent network. With the proposed AEROMamba, training requires 2-4x less GPU memory, since Mamba exploits the convolutional formulation and leverages GPU memory hierarchy. Additionally, during inference, Mamba operates in constant memory due to recurrence, avoiding memory growth associated with Attention. This results in a 14x speed improvement using 5x less GPU. Subjective listening tests (0 to 100 scale) show that the proposed model surpasses the AERO model. In the MUSDB dataset, degraded signals scored 38.22, while AERO and AEROMamba scored 60.03 and 66.74, respectively. For the PianoEval dataset, scores were 72.92 for degraded signals, 76.89 for AERO, and 84.41 for AEROMamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。