动态切换音视频生成模式,实现卫星通信下的高清同步传输
Semantic Satellite Communications for Synchronized Audiovisual Reconstruction
- 根据信道状态灵活切换音/视频生成主导权,只传关键模态
- 仿真显示带宽消耗显著降低,音视频同步质量高且鲁棒性强
- 结合大模型决策,自动适配天气与用户需求,提升系统智能性
卫星通信在支持高质量音视频同步服务时面临严峻挑战,传统方案在信道波动、带宽受限和长延迟条件下难以保持跨模态一致性。本文提出一种面向卫星场景的自适应多模态语义传输系统,旨在带宽受限下实现高保真音视频同步重建。不同于固定优先级的静态方案,该框架采用双流生成架构,可灵活切换视频驱动音频生成或音频驱动视频生成,动态解耦语义,仅传输核心模态并利用跨模态生成恢复缺失内容。为平衡重建质量与传输开销,设计了动态关键帧更新机制,根据无线环境和用户需求自适应维护共享知识库。进一步引入基于大语言模型的决策模块,融合卫星特有知识,联合考虑任务需求与天气引起的信道衰落,主动调整传输路径与生成流程。仿真结果表明,所提系统显著降低带宽消耗,同时实现高保真音视频同步,在复杂卫星场景中提升传输效率与鲁棒性。
原文摘要 · Abstract (English)
Satellite communications face severe bottlenecks in supporting high-fidelity synchronized audiovisual services, as conventional schemes struggle with cross-modal coherence under fluctuating channel conditions, limited bandwidth, and long propagation delays. To address these limitations, this paper proposes an adaptive multimodal semantic transmission system tailored for satellite scenarios, aiming for high-quality synchronized audiovisual reconstruction under bandwidth constraints. Unlike static schemes with fixed modal priorities, our framework features a dual-stream generative architecture that flexibly switches between video-driven audio generation and audio-driven video generation. This allows the system to dynamically decouple semantics, transmitting only the most important modality while employing cross-modal generation to recover the other. To balance reconstruction quality and transmission overhead, a dynamic keyframe update mechanism adaptively maintains the shared knowledge base according to wireless scenarios and user requirements. Furthermore, a large language model based decision module is introduced to enhance system adaptability. By integrating satellite-specific knowledge, this module jointly considers task requirements and channel factors such as weather-induced fading to proactively adjust transmission paths and generation workflows. Simulation results demonstrate that the proposed system significantly reduces bandwidth consumption while achieving high-fidelity audiovisual synchronization, improving transmission efficiency and robustness in challenging satellite scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。