将多个模型合并为单一图结构，提升边缘设备上模型切换效率
FusedInf: Efficient Swapping of DNN Models for On-Demand Serverless Inference Services on the Edge
- 通过构建统一有向无环图整合多个模型
- 实测执行速度最高快14%，内存占用降低17%
- 适合资源受限的边缘推理服务场景
边缘AI计算设备作为新兴计算装置，正推动AI产业变革。这类紧凑可靠的硬件将AI处理能力直接带至网络边缘的数据源端。与此同时，按需提供的无服务器推理服务日益流行，可显著降低中小企业的基础设施成本。然而，这些设备在资源可用性方面仍受限制，服务提供商需高效加载与卸载模型以应对不断增长的需求。本文提出FusedInf，用于在边缘实现高效的按需无服务器推理模型交换。FusedInf将多个模型融合为单一有向无环图（DAG），从而更高效地将模型加载至GPU内存并加快执行速度。对多个主流DNN模型的评估显示，构建单一DAG可使模型执行速度最高提升14%，同时内存需求最多减少17%。原型系统已在https://github.com/SifatTaj/FusedInf发布。
原文摘要 · Abstract (English)
Edge AI computing boxes are a new class of computing devices that are aimed to revolutionize the AI industry. These compact and robust hardware units bring the power of AI processing directly to the source of data--on the edge of the network. On the other hand, on-demand serverless inference services are becoming more and more popular as they minimize the infrastructural cost associated with hosting and running DNN models for small to medium-sized businesses. However, these computing devices are still constrained in terms of resource availability. As such, the service providers need to load and unload models efficiently in order to meet the growing demand. In this paper, we introduce FusedInf to efficiently swap DNN models for on-demand serverless inference services on the edge. FusedInf combines multiple models into a single Direct Acyclic Graph (DAG) to efficiently load the models into the GPU memory and make execution faster. Our evaluation of popular DNN models showed that creating a single DAG can make the execution of the models up to 14\% faster while reducing the memory requirement by up to 17\%. The prototype implementation is available at https://github.com/SifatTaj/FusedInf.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。