用记忆增强模型提升低信噪比下的文本编码鲁棒性
Contextual Memory-Enhanced Source Coding for Low-SNR Communications
- 引入上下文记忆模块，动态存储多阶词组模式
- 通过专家路由机制自适应优化概率估计，码长缩短23%
- 适合通信系统中对错误敏感的文本传输场景
分离信源信道编码（SSCC）在噪声环境下文本传输时仍易出错，尤其当算术编码依赖大语言模型的概率估计时，自回归解码过程脆弱。本文提出一种记忆增强型信源编码（MASC）方案，将上下文模式内化至信源模型中。具体而言，MASC采用共享的参数化上下文记忆（PCM）存储多阶n-gram模式，并设计混合记忆专家路由器（MMER），实现基于隐状态的稀疏、动态记忆专家路由。该机制可自适应优化概率估计，缩短码长，降低解码对残留信道误差的敏感性。在瑞利衰落与加性高斯白噪声（AWGN）信道上的实验验证了其有效性。
原文摘要 · Abstract (English)
Separate Source-Channel Coding (SSCC) remains vulnerable in noisy text transmission due to the fragility of autoregressive source decoding, especially when Arithmetic Coding (AC) relies on Large Language Model (LLM)-based probability estimation. This letter proposes a Memory-Augmented Source Coding (MASC) scheme that internalizes contextual patterns into a source model. Specifically, MASC employs a shared Parameterized Contextual Memory (PCM) for multi-order $n$-gram patterns, and a Mixture-of-Memory-Experts Router (MMER) for sparse, hidden-state-dependent routing over memory experts. This adaptive activation refines source probability estimation, shortens codelength, and mitigates decoding sensitivity to residual channel errors. Experiments over Rayleigh fading and AWGN channels demonstrate its effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。