arXiv:2501.15414eess.IV2025-01被引 2

根据信道和内容熵动态调整传输速率,提升通信效率。

Entropy-and-Channel-Aware Adaptive-Rate Semantic Communication with MLLM-Aided Feature Compensation

  • 结合信道状态与信号噪声比,动态控制特征图和符号的传输
  • 在恶劣信道下多传资源,良好信道下少传仍保高性能
  • 用大模型补偿丢弃信息,实现高效低耗的智能语义通信

尽管语义通信(SemCom)相比传统方法提升了传输效率,但现有大多数方案仍采用固定传输速率,未随信道条件和内容变化调整,导致在有利信道中资源浪费、在恶劣信道中性能下降。为此,本文提出一种新型语义通信框架,针对MIMO瑞利衰落信道设计了基于熵与信道感知的自适应速率控制机制。具体地,将信道状态信息(CSI)与信噪比(SNR)联合表征嵌入语义编码器与解码器,实现信道感知的语义编解码。进一步,通过两个策略网络联合利用CSI、SNR、特征图及其二维熵,选择性传输部分特征图和其中的部分符号,实现比现有方法更细粒度的自适应速率控制。在接收端,借助多模态大语言模型(MLLM)强大的视觉理解能力,采用预训练模型InternVL3.5中的轻量级视觉编码器InternViT-300M,对被丢弃的特征图与符号进行补偿,并使用低秩适配(LoRA)进行参数高效的微调。实验表明,通过精心设计的信道感知损失函数,系统可在较差信道下自动分配更多通信资源以提升任务性能,而在良好信道下减少资源消耗同时维持高任务性能。

原文摘要 · Abstract (English)

Despite the transmission efficiency gains of semantic communication (SemCom) over traditional methods, most existing SemCom schemes still operate at a fixed transmission rate regardless of channel conditions and transmitted content, resulting in wasted resources in favorable channels and degraded performance in harsh channels. To address this issue, we propose a novel SemCom framework that incorporates an entropy-and-channel-aware adaptive rate control mechanism over MIMO Rayleigh fading channels. Specifically, we embed a joint representation of the channel state information (CSI) and the signal-to-noise ratio (SNR) into both the semantic encoder and decoder, thereby realizing channel-aware semantic coding and decoding. Moreover, the proposed method jointly exploits the CSI, the SNR, the feature maps, and their 2D entropy via two policy networks to selectively transmit only a subset of feature maps and, within each selected feature map, only a subset of symbols. Thereby, it achieves finer-grained adaptive rate control than existing methods. At the receiver, leveraging the strong visual understanding capability of multimodal large language models (MLLMs), we deploy the lightweight visual encoder (InternViT-300M) of the pre-trained InternVL3.5 model to compensate for discarded feature maps and symbols, and we fine-tune InternViT using low-rank adaptation (LoRA) for parameter-efficient training. Experimental results show that, with a carefully designed channel-aware loss function, our system automatically allocates more communication resources under poor channels to enhance task performance while reducing resource usage under favorable channels and maintaining high task performance.

语义通信自适应速率多模态大模型信道感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。