用大模型提升跨模态通信效率，让信息传输更智能
Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communications
- 以文本令牌为通信单元，利用大模型处理跨模态上下文
- 图像语义通信中带宽效率显著提升，具体数值未给出
- 适合对通信智能化和效率有要求的未来无线系统研究者
本文提出一种基于大模型驱动的跨模态上下文感知语义通信新框架——令牌通信（TokCom）。该框架受生成式基础模型与多模态大语言模型（GFM/MLLMs）成功启发，将通信单元定义为令牌，使收发端可高效进行基于Transformer的令牌处理。本文探讨了在生成式语义通信（GenSC）中利用上下文信息的机遇与挑战，研究如何将基于GFM/MLLMs的令牌处理集成到语义通信系统中，在可接受复杂度下有效利用跨模态上下文，并提出未来无线网络各层实现高效TokCom的关键原则。在典型图像语义通信场景中，实验证明了通过利用令牌间的上下文信息，可显著提升带宽效率。最后，识别出推动TokCom在下一代无线网络中应用的潜在研究方向。
原文摘要 · Abstract (English)
In this paper, we introduce token communications (TokCom), a large model-driven framework to leverage cross-modal context information in generative semantic communications (GenSC). TokCom is a new paradigm, motivated by the recent success of generative foundation models and multimodal large language models (GFM/MLLMs), where the communication units are tokens, enabling efficient transformer-based token processing at the transmitter and receiver. In this paper, we introduce the potential opportunities and challenges of leveraging context in GenSC, explore how to integrate GFM/MLLMs-based token processing into semantic communication systems to leverage cross-modal context effectively at affordable complexity, present the key principles for efficient TokCom at various layers in future wireless networks. In a typical image semantic communication setup, we demonstrate a significant improvement of the bandwidth efficiency, achieved by TokCom by leveraging the context information among tokens. Finally, the potential research directions are identified to facilitate adoption of TokCom in future wireless networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。