用设备端语义分解实现实时生成式多播，节省大量网络资源。
Demo: Real-time Generative Multicasting with On-Device Intent-aware Semantic Decomposition
- 基于深度网络提取视频语义图，按用户意图拆分信号
- 实测在4 TOPS边缘设备上实时运行，显著降低传输资源需求
- 适合低带宽场景下的个性化多媒体多播应用
我们展示了一种基于设备端、意图感知的语义分解生成式多播系统。发送端通过深度神经网络对源视频进行分割，生成语义图，并根据多用户接收意图将信号分解为多个子信号类别。发送端仅通过共享无线/网络资源广播语义图，仅对每个用户的特定子信号类别使用正交资源传输。接收端结合接收到的意图内类别与本地由语义图生成模型合成的非意图类别，部分重建并部分合成信号。我们推导了生成模型下重构/合成的率失真/感知曲线，以自适应设定语义图和意图类别的压缩率。该生成式多播显著降低了现有及新兴多媒体多播应用所需的无线/网络资源。系统在支持4 TOPS（int8）的Google Coral Edge TPU上实现实时运行。这是首个生成式多播演示，标志着设备端生成式语义通信的重大进展。
原文摘要 · Abstract (English)
We present a demonstration for generative multicasting with on-device, intent-aware semantic decomposition. At the transmitter, DNN-based segmentation extracts a semantic map from the source video, decomposing it into multiple sub-signal classes based on multi-user receiver intents. The transmitter broadcasts the semantic map to all users over shared wireless/network resources, thereby utilizing orthogonal resources only to transmit the sub-signal classes intended for each user. Users partially reconstruct and partially synthesize the signal by combining the received intended classes with non-intended classes locally synthesized by a generative model from the semantic map. We derive the rate-distortion/perception curves for reconstruction/synthesis with the generative model, to adaptively set compression rates for the semantic map and intended classes. Generative multicasting significantly reduces the wireless/network resources required for existing/emerging multimedia multicasting applications. The system is real-time on a Google Coral Edge TPU with 4 TOPS (int8). This is the first demonstration of generative multicasting representing a substantial advancement in on-device generative SemCom.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。