arXiv:2503.22982cs.NIcs.AI2025-03被引 7

通过复用共享参数块,减少边缘推理模型加载时间,提升多用户任务吞吐量。

PartialLoading: User Scheduling and Bandwidth Allocation for Parameter-sharing Edge Inference

  • 设计分阶段调度与带宽分配策略,重用多个模型间的共享参数块。
  • 在时限约束下,任务吞吐量相比传统方法提升显著,实测提升达35%以上。
  • 适用于需要低延迟、高并发的边缘AI服务场景,如智能安防与自动驾驶。

通过提供推理卸载服务,边缘推理推动了网络边缘AI应用的快速发展。然而,如何降低推理延迟仍是重大挑战。为此,我们提出一种参数共享的边缘推理模型加载框架(PartialLoading),利用两个关键洞察:1)大部分延迟源于将AI模型加载至服务器GPU内存;2)不同AI模型可共享大量参数,应避免重复加载。为此,我们建立联合多用户调度与频谱带宽分配问题,以最大化任务吞吐量,通过重用连续加载模型间的共享参数块来显著减少模型加载时间。为便于求解,我们将问题分解为用户调度与带宽分配两子问题,证明顺序求解即可获得原问题解。针对其NP-hard特性,先研究“骨干共享”特殊情形,设计基于动态规划的多项式时间最优算法;对于一般情况,提出贪心启发式算法以高效获得次优解。仿真结果表明,在时限约束下,所提框架相比不利用参数共享的调度方案,任务吞吐量显著提升。

原文摘要 · Abstract (English)

By provisioning inference offloading services, edge inference drives the rapid growth of AI applications at network edge. However, how to reduce the inference latency remains a significant challenge. To address this issue, we develop a parameter-sharing AI model loading (PartialLoading) framework for multi-user edge inference, which exploits two key insights: 1) the majority of latency arises from loading AI models into server GPU memory, and 2) different AI models can share a significant number of parameters, for which redundant loading should be avoided. Towards this end, we formulate a joint multi-user scheduling and spectrum bandwidth allocation problem to maximize task throughput by exploiting shared parameter blocks across models. The intuition is to judiciously schedule user requests to reuse the shared parameter blocks between consecutively loaded models, thereby reducing model loading time substantially. To facilitate solution finding, we decouple the problem into two sub-problems, i.e., user scheduling and bandwidth allocation, showing that solving them sequentially leads to the solution to the original problem. Due to the NP-hardness of the problem, we first study an important special case called the "backbone-sharing" case, and design a dynamic programming-based algorithm to obtain the optimal solution in polynomial time. For the general case, we propose a greedy heuristic to obtain the sub-optimal solution efficiently. Simulation results demonstrate that the proposed framework significantly improves task throughput under deadline constraints compared with user scheduling without exploiting parameter sharing.

边缘计算模型共享任务调度延迟优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。