将状态空间模型转化为张量运算，提升训练效率与设计灵活性。
Let SSMs be ConvNets: State-space Modeling with Optimal Tensor Contractions
- 将SSM操作视为张量收缩，优化计算顺序以提高训练效率。
- 在语音识别等任务中性能超越传统同质化结构，无需依赖LSTM或注意力机制。
- 融合卷积网络设计思想，兼顾模型大小、内存与计算效率，适合音频处理场景。
我们提出Centaurus，一类由广义状态空间模型（SSM）块组成的网络，其中SSM运算在训练过程中可被视为张量收缩。通过系统性地确定每个SSM块的最优张量收缩顺序，可最大化训练效率。这使得设计更灵活，突破了常见的深度可分离结构限制，借鉴经典卷积块如分组卷积、全卷积和瓶颈结构。我们构建了包含这些异构块的Centaurus网络，在训练与推理阶段均平衡了模型规模、性能、内存与计算效率。实验表明，该网络在原始音频处理任务（关键词检测、语音去噪、自动语音识别，即ASR）中优于同质化结构。在ASR任务中，Centaurus是首个实现竞争力性能且完全基于状态空间的模型，无需非线性递归（如LSTM）、显式卷积（CNN）或（近似）注意力机制。源代码作为补充材料发布于https://openreview.net/forum?id=PkpNRmBZ32。
原文摘要 · Abstract (English)
We introduce Centaurus, a class of networks composed of generalized state-space model (SSM) blocks, where the SSM operations can be treated as tensor contractions during training. The optimal order of tensor contractions can then be systematically determined for every SSM block to maximize training efficiency. This allows more flexibility in designing SSM blocks beyond the depthwise-separable configuration commonly implemented. The new design choices will take inspiration from classical convolutional blocks including group convolutions, full convolutions, and bottleneck blocks. We architect the Centaurus network with a mixture of these blocks, to balance between network size and performance, as well as memory and computational efficiency during both training and inference. We show that this heterogeneous network design outperforms its homogeneous counterparts in raw audio processing tasks including keyword spotting, speech denoising, and automatic speech recognition (ASR). For ASR, Centaurus is the first network with competitive performance that can be made fully state-space based, without using any nonlinear recurrence (LSTMs), explicit convolutions (CNNs), or (surrogate) attention mechanism. The source code is available as supplementary material on https://openreview.net/forum?id=PkpNRmBZ32
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。