用轻量化AI提升语义通信效率,实时性强且自适应信道变化。
Lightweight Task-Oriented Semantic Communication Empowered by Large-Scale AI Models
- 通过预存压缩机制与快速蒸馏,减少重复推理开销。
- 在低延迟、小模型规模下实现更高任务准确率和更低数据需求。
- 支持动态调整传输内容以适应不同信道,适合移动实时通信场景。
近期研究致力于利用大规模人工智能(LAI)模型提升语义表征与压缩能力,但其高计算需求限制了实时通信应用。为此,本文提出基于知识蒸馏(KD)的方法,从LAI模型中提取并压缩知识,显著降低模型复杂度与计算延迟。然而,LAI模型的固有复杂性导致蒸馏过程推理时间长,且缺乏信道感知能力,影响蒸馏效果。为此,我们设计一种快速蒸馏方法,引入预存储压缩机制,避免重复推理,大幅提高效率;同时集成信道自适应模块,根据信道状态动态调整传输语义信息,增强通信可靠性与适应性。此外,提出基于信息瓶颈的损失函数,指导快速蒸馏过程。仿真结果表明,所提方案在任务准确率、模型大小、计算延迟及训练数据需求方面均优于基线方法。
原文摘要 · Abstract (English)
Recent studies have focused on leveraging large-scale artificial intelligence (LAI) models to improve semantic representation and compression capabilities. However, the substantial computational demands of LAI models pose significant challenges for real-time communication scenarios. To address this, this paper proposes utilizing knowledge distillation (KD) techniques to extract and condense knowledge from LAI models, effectively reducing model complexity and computation latency. Nevertheless, the inherent complexity of LAI models leads to prolonged inference times during distillation, while their lack of channel awareness compromises the distillation performance. These limitations make standard KD methods unsuitable for task-oriented semantic communication scenarios. To address these issues, we propose a fast distillation method featuring a pre-stored compression mechanism that eliminates the need for repetitive inference, significantly improving efficiency. Furthermore, a channel adaptive module is incorporated to dynamically adjust the transmitted semantic information based on varying channel conditions, enhancing communication reliability and adaptability. In addition, an information bottleneck-based loss function is derived to guide the fast distillation process. Simulation results verify that the proposed scheme outperform baselines in term of task accuracy, model size, computation latency, and training data requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。