无需操作系统,直接操控硬件实现高效AI推理。
AEG: A Baremetal Framework for AI Acceleration via Direct Hardware Access in Heterogeneous Accelerators

- 用线性控制块替代复杂逻辑,实现跨硬件的统一运行时。
- 相比传统方案,算力效率提升9.2倍,数据移动开销减少3-7倍。
- 适合对延迟敏感、资源受限的边缘AI部署场景。
本文提出一种统一的裸金属运行时架构,可在异构加速器(如AI Engine阵列)上实现高性能机器学习推理,无需依赖实时或通用操作系统。现有边缘部署框架(如TinyML)常依赖实时操作系统,引入额外复杂性和性能瓶颈。为此,本方案通过将复杂控制逻辑扁平化为可执行的运行时控制块(RCBs),实现运行时与硬件的解耦,采用“控制即数据”范式,使高级模型(如自适应数据流图)可通过最小化的运行时硬件抽象层(RHAL)由通用引擎执行。同时集成运行时平台管理(RTPM)处理系统级调度(含轻量级网络栈)和运行时内存文件系统(RIMFS)以管理无操作系统环境中的数据。通过ResNet-18图像分类实现验证:相比基于Linux的Vitis AI部署,计算效率提升9.2倍(每AIE单元吞吐量),数据移动开销降低3–7倍,延迟方差趋近于零(变异系数CV~0.03%)。仅使用28个AIE单元即在ImageNet上达到68.78% Top-1准确率,远低于Vitis AI所需304个单元,验证了该统一裸金属架构的高效性与正确性。
原文摘要 · Abstract (English)
This paper introduces a unified, hardware-independent baremetal runtime architecture designed to enable high-performance machine learning (ML) inference on heterogeneous accelerators, such as AI Engine (AIE) arrays, without the overhead of an underlying real-time or general-purpose operating system. Existing edge-deployment frameworks, such as TinyML, often rely on real-time operating systems (RTOS), which introduce unnecessary complexity and performance bottlenecks. To address this, our solution fundamentally decouples the runtime from hardware specifics by flattening complex control logic into linear, executable Runtime Control Blocks (RCBs). This "Control as Data" paradigm allows high-level models, including Adaptive Data Flow (ADF) graphs, to be executed by a generic engine through a minimal Runtime Hardware Abstraction Layer (RHAL). We further integrate Runtime Platform Management (RTPM) to handle system-level orchestration (including a lightweight network stack) and a Runtime In-Memory File System (RIMFS) to manage data in OS-free environments. We demonstrate the framework's efficacy with a ResNet-18 image classification implementation. Experimental results show 9.2$\times$ higher compute efficiency (throughput per AIE tile) compared to Linux-based Vitis AI deployment, 3--7$\times$ reduction in data movement overhead, and near-zero latency variance (CV~$=0.03\%$). The system achieves 68.78\% Top-1 accuracy on ImageNet using only 28 AIE tiles compared to Vitis AI's 304 tiles, validating both the efficiency and correctness of this unified bare-metal architecture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。