高丢包下视频语义通信新方案，用混合专家模型提升鲁棒性
Conquering High Packet-Loss Erasure: MoE Swin Transformer-Based Video Semantic Communication
- 采用MoE Swin Transformer编码语义，结合3D CNN与丢包掩码恢复缺失信息
- 在90%丢包率下仍保持MS-SSIM>0.6、PSNR>20dB的优异表现
- 适合低带宽、高丢包场景的实时视频语义传输，如远程医疗、车载通信
面向联合语义-信道编码的语义通信系统，在基于分组的传输中面临因数据包丢失导致语义信息损失的问题。现有协议中错误数据包被直接丢弃，无法用于接收端鲁棒解码。为此，本文提出一种抗丢包的MoE Swin Transformer视频语义通信系统（MSTVSC），通过上层协议对语义向量进行分组传输。为研究分组策略影响，提供了理论分析。为缓解丢包造成的语义损失，接收端引入3D CNN，利用未丢失的语义数据和丢包掩码矩阵重建缺失内容。采用语义级交织降低集中丢包带来的损失。为提升压缩效率，采用公共-个体分解方法，并对个体信息下采样以减少冗余。模型轻量化设计便于实际部署。大量仿真与对比实验表明，该系统在90%丢包率下仍能实现MS-SSIM > 0.6、PSNR > 20 dB的性能。
原文摘要 · Abstract (English)
Semantic communication with joint semantic-channel coding robustly transmits diverse data modalities but faces challenges in mitigating semantic information loss due to packet drops in packet-based systems. Under current protocols, packets with errors are discarded, preventing the receiver from utilizing erroneous semantic data for robust decoding. To address this issue, a packet-loss-resistant MoE Swin Transformer-based Video Semantic Communication (MSTVSC) system is proposed in this paper. Semantic vectors are encoded by MSTVSC and transmitted through upper-layer protocol packetization. To investigate the impact of the packetization, a theoretical analysis of the packetization strategy is provided. To mitigate the semantic loss caused by packet loss, a 3D CNN at the receiver recovers missing information using un-lost semantic data and an packet-loss mask matrix. Semantic-level interleaving is employed to reduce concentrated semantic loss from packet drops. To improve compression, a common-individual decomposition approach is adopted, with downsampling applied to individual information to minimize redundancy. The model is lightweighted for practical deployment. Extensive simulations and comparisons demonstrate strong performance, achieving an MS-SSIM greater than 0.6 and a PSNR exceeding 20 dB at a 90% packet loss rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。