arXiv:2503.08451cs.ITcs.LG2025-03ICML被引 1

用轻量GRU和Mamba块提升长序列信道编码的效率与可扩展性。

MinGRU-Based Encoder for Turbo Autoencoder Frameworks

  • 将minGRU与Mamba块融合,构建并行化神经信道编码框架。
  • 在长码长下训练效率提升3倍,性能接近传统卷积方法。
  • 适合需要高可扩展性的长序列通信系统设计者参考。

早期神经信道编码方法采用全连接网络与独热编码设计自适应编解码对,改善了块错误率(BLER)并实现了自动化设计,但随着消息集规模和码块长度增加,其可扩展性受限。TurboAE通过处理比特序列而非符号级表示,将大规模消息集带来的可扩展性问题转化为序列建模问题。虽然循环神经网络(RNN)天然适用于序列处理,但其串行计算导致长序列时计算开销大、效率低。因此,TurboAE采用卷积网络模块,虽训练更快且更易扩展,却失去了RNN的序列建模优势。近年来,高效RNN架构如minGRU、minLSTM以及结构化状态空间模型(SSM)如S4、S6,通过大幅降低内存与计算开销,克服了上述限制,使RNN在长序列任务中重新具备竞争力。本文重新审视在Turbo自动编码器中使用RNN,将轻量级minGRU与来自SSM的Mamba块集成到并行化Turbo自动编码框架中。实验表明,该混合设计在短序列上性能与基于卷积的TurboAE相当,而在长码长下显著提升可扩展性与训练效率,凸显了高效RNN在长序列神经信道编码中的潜力。

原文摘要 · Abstract (English)

Early neural channel coding approaches leveraged dense neural networks with one-hot encodings to design adaptive encoder-decoder pairs, improving block error rate (BLER) and automating the design process. However, these methods struggled with scalability as the size of message sets and block lengths increased. TurboAE addressed this challenge by focusing on bit-sequence inputs rather than symbol-level representations, transforming the scalability issue associated with large message sets into a sequence modeling problem. While recurrent neural networks (RNNs) were a natural fit for sequence processing, their reliance on sequential computations made them computationally expensive and inefficient for long sequences. As a result, TurboAE adopted convolutional network blocks, which were faster to train and more scalable, but lacked the sequential modeling advantages of RNNs. Recent advances in efficient RNN architectures, such as minGRU and minLSTM, and structured state space models (SSMs) like S4 and S6, overcome these limitations by significantly reducing memory and computational overhead. These models enable scalable sequence processing, making RNNs competitive for long-sequence tasks. In this work, we revisit RNNs for Turbo autoencoders by integrating the lightweight minGRU model with a Mamba block from SSMs into a parallel Turbo autoencoder framework. Our results demonstrate that this hybrid design matches the performance of convolutional network-based Turbo autoencoder approaches for short sequences while significantly improving scalability and training efficiency for long block lengths. This highlights the potential of efficient RNNs in advancing neural channel coding for long-sequence scenarios.

信道编码RNNMamba并行计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。