用机器共感模型提升复杂任务下图像的可靠传输
Synesthesia of Machines (SoM)-Based Task-Driven MIMO System for Image Transmission
- 基于特征金字塔与闭环MIMO结构设计任务驱动编码
- 在相同开销下，平均mAP提升6.30至10.48
- 适合动态场景中移动智能体的协同感知应用
为支持动态场景中联网移动智能体的协同感知（CP），高效可靠的感官数据传输是关键挑战。基于深度学习的联合源信道编码（JSCC）在恶劣信道条件下表现优异，超越传统规则编码器。尽管近期研究尝试将JSCC与广泛采用的多输入多输出（MIMO）技术结合，但现有方法仍局限于离散时间模拟传输（DTAT）模型和简单任务。针对数字MIMO通信系统中现有MIMO JSCC方案在复杂协同感知任务上的性能不足，本文提出一种基于机器共感（SoM）的任务驱动MIMO图像传输系统，称为SoM-MIMO。通过利用感知任务的特征金字塔结构特性与闭环MIMO通信系统的信道特性，SoM-MIMO实现了高效且鲁棒的数字MIMO图像传输。实验结果表明，在所有信噪比（SNR）水平下，相比两种JSCC基线方案，本方法平均mAP分别提升6.30和10.48，同时保持相同的通信开销。
原文摘要 · Abstract (English)
To support cooperative perception (CP) of networked mobile agents in dynamic scenarios, the efficient and robust transmission of sensory data is a critical challenge. Deep learning-based joint source-channel coding (JSCC) has demonstrated promising results for image transmission under adverse channel conditions, outperforming traditional rule-based codecs. While recent works have explored to combine JSCC with the widely adopted multiple-input multiple-output (MIMO) technology, these approaches are still limited to the discrete-time analog transmission (DTAT) model and simple tasks. Given the limited performance of existing MIMO JSCC schemes in supporting complex CP tasks for networked mobile agents with digital MIMO communication systems, this paper presents a Synesthesia of Machines (SoM)-based task-driven MIMO system for image transmission, referred to as SoM-MIMO. By leveraging the structural properties of the feature pyramid for perceptual tasks and the channel properties of the closed-loop MIMO communication system, SoM-MIMO enables efficient and robust digital MIMO transmission of images. Experimental results have shown that compared with two JSCC baseline schemes, our approach achieves average mAP improvements of 6.30 and 10.48 across all SNR levels, while maintaining identical communication overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。