arXiv:2605.07354eess.SPcs.CV2026-05

用边缘-云协同压缩动作数据,传输量减至1%,延迟降为20%。

Task-Oriented Communication for Human Action Understanding via Edge-Cloud Co-Inference

论文配图:Task-Oriented Communication for Human Action Understanding via Edge-Cloud Co-Inference
图 1 · 摘自论文原文
  • 只传动作编码索引,每帧仅9比特,避免原始视频上传
  • 在云端用大模型理解动作,准确率与传统方法相当
  • 适合低带宽、高隐私要求的实时人体动作识别场景

智能感知应用的扩展带来了对网络边缘人类动作理解的日益增长需求。传统方法需将大量视频数据从资源受限的边缘设备传输至强大的云服务器,导致上行带宽消耗巨大、延迟不可接受,并引发隐私担忧。为克服这些瓶颈,我们提出一种面向任务的通信框架(TOAU),通过边缘-云协同实现人类动作理解。该框架利用单目姿态估计算法从原始视频中提取连续关节坐标,再通过向量量化变分自编码器(VQ-VAE)将这些坐标转换为离散动作标记。因此,仅需在网络上传输紧凑的动作标记序列,每帧仅消耗9比特,避免隐私泄露。在云服务器端,一个轻量级投影器将这些动作标记对齐至大型视觉语言模型(VLM)的嵌入空间,以支持复杂动作理解,该模型采用高效的指令微调范式进行训练。在三个基准上的综合评估表明,与基于视频编码的解决方案相比,我们的TOAU系统将传输负载降低约1%,系统延迟降至约20%,同时保持相近的动作理解准确率。

原文摘要 · Abstract (English)

The expanding application of smart sensing has created a growing demand for the accurate understanding of human action at the network edge. Traditional approaches require massive video data to be transmitted from resource-constrained edge devices to powerful cloud servers, incurring prohibitive uplink bandwidth consumption and unacceptable latency while raising privacy concerns. To overcome these bottlenecks, we propose a task-oriented communication framework for human action understanding (TOAU) through edge-cloud collaboration. Our framework utilizes a monocular pose estimator to extract continuous joint coordinates from raw videos, followed by a vector quantized variational autoencoder (VQ-VAE) to convert these coordinates into discrete motion tokens. Consequently, only a compact sequence of codebook indices is transmitted over the network, consuming as few as 9 bits per frame and avoiding privacy leakages. At the cloud server, a lightweight projector aligns these motion tokens with the embedding space of a large vision-language model (VLM) to facilitate complex action understanding, which is trained with an efficient instruction tuning paradigm. Comprehensive evaluations on three benchmarks demonstrate that our TOAU system reduces the transmission payload to approximately 1\% and the system latency to around 20\% compared to video codec-based solutions, while delivering comparable action understanding accuracy.

边缘计算动作识别通信优化隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。