用通用近似技术挖潜边缘AI芯片闲置算力,提升整体效率。
Harvesting AI Computation at the Edge via Generic Approximation
- 通过神经架构搜索将通用任务转为神经网络模型
- 在AI芯片空闲时调度执行,性能提升显著
- 适合资源受限的边缘AIoT场景
随着AI在智能传感与处理等物联网场景中的广泛应用,边缘端的AI芯片已成为标配。这些芯片通常专用于结构化神经网络处理,设计时按峰值负载规划,但因处理过程存在时间或空间冗余,常处于低利用率状态,造成大量计算浪费。与此同时,边缘端的通用处理器在面对信号处理、复杂数值运算等高负载任务时,受限于资源紧张而难以胜任。为此,本文提出一种利用通用近似技术挖掘边缘闲置AI算力的框架。核心思路是通过代表性神经架构搜索(NAS)方法,自动将传统通用计算任务转换为神经网络模型,并在AI芯片空闲时段部署运行。我们设计了一种运行时调度器,在不干扰主AI任务的前提下,将这些近似任务卸载至AI芯片,从而减轻通用处理器负担。在典型AIoT处理器上的实验表明,该策略在多项边缘处理任务中均实现显著性能提升。
原文摘要 · Abstract (English)
With the widespread adoption of AI in various IoT scenarios such as smart sensing and processing, AI chips have become a common component at the edge. These chips are typically specialized for structured neural network (NN) processing and are designed to meet peak workload demands. However, they are often underutilized and suffer from considerable computational waste due to temporal or spatial redundancy in processing. Conversely, general-purpose processing engines at the edge may struggle with compute-intensive tasks such as signal processing and complex numerical operations because of stringent resource constraints. To address this imbalance, we propose a framework that harvests unused AI computation resources using general-purpose approximation techniques. The core idea is to automatically convert traditional computing tasks into neural network models via a representative neural architecture search (NAS) method. These approximate versions of general-purpose tasks are then deployed on AI engines during their idle periods. Specifically, we introduce a runtime scheduler that offloads these tasks to AI chips without compromising the performance of primary AI workloads, thereby alleviating the burden on general-purpose processors. Experiments on a representative AIoT processor show that our proposed AI computation harvesting strategy delivers substantial performance improvements across a set of edge processing tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。