针对IoVT系统实时视觉推理,提出软硬件协同优化框架提升性能与适应性。
Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment
- 基于重参数化动态结构与屋顶模型分区策略,提升边缘设备计算效率。
- 在MNIST和ImageNet上分别实现12.05%和18.83%的吞吐率提升,精度更优。
- 适用于多类型边缘设备,适合对实时性与准确率要求高的物联网视觉场景。
随着图像数据量增长,传统面向数据的云计算在物联网视频系统(IoVT)中面临延迟问题。任务导向的边缘计算通过将数据分析移至边缘缓解此问题,但边缘设备算力有限,难以高效执行视觉任务。现有方法难以兼顾高模型性能与低资源消耗:轻量级神经网络性能不足,而由神经架构搜索(NAS)设计的设备专用模型又无法适应异构设备。为此,我们提出一种新型软硬件协同设计框架,在推理阶段联合优化神经网络结构与部署策略,以实现高吞吐。具体而言,该框架采用基于重参数化的动态模型结构,并结合基于屋顶模型(Roofline)的模型分片策略,提升边缘设备计算性能;同时使用多目标协同优化方法平衡吞吐与精度。此外,推导了分片模型的数学一致性与收敛性。实验表明,相比基线算法,本方法在MNIST上吞吐提升12.05%,在ImageNet上提升18.83%,分类精度更优。该方法在不同设备上均保持稳定性能,展现出良好适应性。模拟实验进一步验证其在小物体高精度实时检测中的有效性。
原文摘要 · Abstract (English)
As the volume of image data grows, data-oriented cloud computing in Internet of Video Things (IoVT) systems encounters latency issues. Task-oriented edge computing addresses this by shifting data analysis to the edge. However, limited computational power of edge devices poses challenges for executing visual tasks. Existing methods struggle to balance high model performance with low resource consumption; lightweight neural networks often underperform, while device-specific models designed by Neural Architecture Search (NAS) fail to adapt to heterogeneous devices. For these issues, we propose a novel co-design framework to optimize neural network architecture and deployment strategies during inference for high-throughput. Specifically, it implements a dynamic model structure based on re-parameterization, coupled with a Roofline-based model partitioning strategy to enhance the computational performance of edge devices. We also employ a multi-objective co-optimization approach to balance throughput and accuracy. Additionally, we derive mathematical consistency and convergence of partitioned models. Experimental results demonstrate significant improvements in throughput (12.05\% on MNIST, 18.83\% on ImageNet) and superior classification accuracy compared to baseline algorithms. Our method consistently achieves stable performance across different devices, underscoring its adaptability. Simulated experiments further confirm its efficacy in high-accuracy, real-time detection for small objects in IoVT systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。