混合Mamba-Transformer解码器显著提升纠错码解码性能
Hybrid Mamba-Transformer Decoder for Error-Correcting Codes
- 结合Mamba序列建模与Transformer全局上下文能力
- 多层掩码策略增强不同层级特征选择能力
- 渐进式分层损失促进中间阶段特征提取
我们提出一种基于Mamba架构的新型深度学习纠错码解码方法，通过引入Transformer层构建混合解码器。该方法利用Mamba的高效序列建模能力，同时保留Transformer的全局上下文感知优势。为进一步提升性能，设计了一种应用于每个Mamba层的层内掩码策略，实现对不同深度下相关码特征的选择性关注。此外，引入渐进式分层损失，在中间阶段监督网络，促进整个解码过程中鲁棒特征提取。在多种线性码上的全面实验表明，该方法显著优于仅使用Transformer的解码器和标准Mamba模型。
原文摘要 · Abstract (English)
We introduce a novel deep learning method for decoding error correction codes based on the Mamba architecture, enhanced with Transformer layers. Our approach proposes a hybrid decoder that leverages Mamba's efficient sequential modeling while maintaining the global context capabilities of Transformers. To further improve performance, we design a novel layer-wise masking strategy applied to each Mamba layer, allowing selective attention to relevant code features at different depths. Additionally, we introduce a progressive layer-wise loss, supervising the network at intermediate stages and promoting robust feature extraction throughout the decoding process. Comprehensive experiments across a range of linear codes demonstrate that our method significantly outperforms Transformer-only decoders and standard Mamba models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。