通过并行计算减少移动端边缘推理传输延迟，提升速度与能效。
Intra-DP: A High Performance Collaborative Inference System for Mobile Edge Computing
- 将卷积等操作拆解为独立子任务并行执行，降低通信瓶颈。
- 相比现有方法，推理延迟降低50%，能耗减少75%。
- 适合资源受限的移动端实时推理场景使用。
在资源受限的移动设备上部署深度神经网络（DNN）面临巨大挑战，尤其在实现实时性能的同时还需应对计算能力不足和电池寿命短的问题。尽管移动边缘计算（MEC）通过将模型分层分配至带GPU的服务器实现协作推理，但现有方法依赖逐层分割，导致因串行执行而产生显著的传输瓶颈。为此，我们提出Intra-DP，一种面向MEC中DNN推理的高性能协作推理系统。Intra-DP采用基于局部算子（即最小输入单位非完整张量的操作，如卷积核）的新型并行计算技术，将计算分解为多个独立子操作，并通过并行执行重叠不同子操作的计算与传输，有效缓解了MEC中的传输瓶颈，实现了快速且节能的推理。评估表明，Intra-DP相比最先进基线，每轮推理延迟降低高达50%，能耗减少高达75%，同时不牺牲准确性。
原文摘要 · Abstract (English)
Deploying deep neural networks (DNNs) on resource-constrained mobile devices presents significant challenges, particularly in achieving real-time performance while simultaneously coping with limited computational resources and battery life. While Mobile Edge Computing (MEC) offers collaborative inference with GPU servers as a promising solution, existing approaches primarily rely on layer-wise model partitioning and undergo significant transmission bottlenecks caused by the sequential execution of DNN operations. To address this challenge, we present Intra-DP, a high-performance collaborative inference system optimized for DNN inference on MEC. Intra DP employs a novel parallel computing technique based on local operators (i.e., operators whose minimum unit input is not the entire input tensor, such as the convolution kernel). By decomposing their computations (operations) into several independent sub-operations and overlapping the computation and transmission of different sub-operations through parallel execution, Intra-DP mitigates transmission bottlenecks in MEC, achieving fast and energy-efficient inference. The evaluation demonstrates that Intra-DP reduces per-inference latency by up to 50% and energy consumption by up to 75% compared to state-of-the-art baselines, without sacrificing accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。