只传主设备没覆盖的多模态信息,大幅降低通信开销。
CST: Collaborative Selective Transmission for Communication-Efficient Multimodal Edge Inference

- 主设备查询时只取互补信息,避免重复传输
- 仅传14.18%的特征值,性能接近最优
- 适合边缘多设备协同推理场景
协同多模态推理通过整合分布式感知设备的观测提升边缘感知能力,但高维辅助表示的传输带来显著通信开销和端到端延迟。现有高效通信方法通过压缩、语义编码或特征选择减少数据量,但通常仅优化紧凑性或任务相关性,未显式考虑主设备已有的信息。因此,即使任务相关也冗余的辅助特征仍会占用带宽。本文提出协作选择性传输(CST),一种主设备主导的查询-响应框架,仅检索与当前主表示互补的辅助信息。受部分信息分解和多视图冗余假设启发,CST学习样本自适应、辅助特定的稀疏检索支撑,同时抑制主设备已有或跨辅助间重复的语义信息获取。推理时,主设备仅发送支撑索引,各辅助返回对应潜在值,避免密集特征交换。在三个真实世界多模态传感基准上,CST传输的辅助特征值不超过14.18%,且性能达到或接近最优。五节点NVIDIA Jetson Orin Nano测试平台在5–100 Mbps条件下,相比全量传输实现最高4.27倍的端到端推理加速,验证了实际延迟降低效果。
原文摘要 · Abstract (English)
Collaborative multimodal inference improves edge perception by combining observations from distributed sensing devices, but transmitting high-dimensional helper representations incurs substantial communication overhead and can lead to high end-to-end latency. Existing communication-efficient methods reduce payloads through compression, semantic coding, or feature selection, yet typically optimize compactness or task relevance without explicitly accounting for information already represented at the main device. Consequently, task-relevant but redundant helper features may still consume bandwidth. We present Collaborative Selective Transmission (CST), a main-directed query--response framework that retrieves only helper information complementary to the current main representation. Inspired by Partial Information Decomposition and the Multiview Redundancy Assumption, CST learns sample-adaptive, helper-specific sparse retrieval supports while discouraging retrieval of semantics already covered by the main device or duplicated across helpers. During inference, the main device transmits only support indices, and each helper returns the corresponding latent values, avoiding dense helper-feature exchange. Across three real-world multimodal sensing benchmarks, CST transmits no more than 14.18% of helper feature values while achieving best or near-best task performance among the evaluated methods. Experiments on a five-node NVIDIA Jetson Orin Nano testbed across 5--100 Mbps demonstrate up to a $4.27\times$ speedup over Transmit-All in end-to-end inference, confirming practical end-to-end latency reductions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。