提出云边协同的多模态3D目标检测特征压缩方法,大幅降低传输量。
Feature Compression for Cloud-Edge Multimodal 3D Object Detection
- 设计两种压缩模式:仅传末层特征(T-FFC)或额外传两类特征(A-FFC)
- T-FFC压缩比达6061倍,性能下降小于3%;A-FFC压缩比约901倍,几乎无损失
- 支持残差提取与3D重建,可还原物体细节,适合工业视觉场景
机器视觉系统在工业生产和日常生活中日益普及。由于单传感器难以同时获取精确深度与纹理信息,常采用相机与LiDAR的多模态数据提升性能。同时,云边协同成为提升用户体验与保障数据安全的新计算范式。本文提出一种面向多模态3D目标检测的特征压缩新方案。针对边缘设备上的稀疏张量检测网络,设计两种模式:传输友好型压缩(T-FFC)和精度友好型压缩(A-FFC)。T-FFC仅传输网络主干末层输出,云端通过通道扩展与双空间上采样生成多尺度特征;A-FFC在此基础上增加两类特征传输,提升云端特征精度。在KITTI数据集上使用VirConv-L检测网络实验表明,T-FFC实现6061倍压缩,性能下降低于3%;A-FFC压缩比约901倍,性能几乎无损。此外,设计了可选残差提取与3D物体重建模块,重构结果能有效反映原始物体细节。
原文摘要 · Abstract (English)
Machine vision systems, which can efficiently manage extensive visual perception tasks, are becoming increasingly popular in industrial production and daily life. Due to the challenge of simultaneously obtaining accurate depth and texture information with a single sensor, multimodal data captured by cameras and LiDAR is commonly used to enhance performance. Additionally, cloud-edge cooperation has emerged as a novel computing approach to improve user experience and ensure data security in machine vision systems. This paper proposes a pioneering solution to address the feature compression problem in multimodal 3D object detection. Given a sparse tensor-based object detection network at the edge device, we introduce two modes to accommodate different application requirements: Transmission-Friendly Feature Compression (T-FFC) and Accuracy-Friendly Feature Compression (A-FFC). In T-FFC mode, only the output of the last layer of the network's backbone is transmitted from the edge device. The received feature is processed at the cloud device through a channel expansion module and two spatial upsampling modules to generate multi-scale features. In A-FFC mode, we expand upon the T-FFC mode by transmitting two additional types of features. These added features enable the cloud device to generate more accurate multi-scale features. Experimental results on the KITTI dataset using the VirConv-L detection network showed that T-FFC was able to compress the features by a factor of 6061 with less than a 3% reduction in detection performance. On the other hand, A-FFC compressed the features by a factor of about 901 with almost no degradation in detection performance. We also designed optional residual extraction and 3D object reconstruction modules to facilitate the reconstruction of detected objects. The reconstructed objects effectively reflected details of the original objects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。