让嵌入式设备运行符号化人工智能,实现高效推理。
Hardware-efficient tractable probabilistic inference for TinyML Neurosymbolic AI applications
- 用硬件友好的确定性概率电路实现符号化模型。
- 通过根号压缩技术将模型缩小至可在微控制器上运行。
- 在ESP32上实现4.67倍加速,资源占用降低超50%。
神经符号人工智能(NSAI)旨在克服深度学习模型在不确定性量化和显式规则推理方面的局限。为将NSAI引入嵌入式场景,TinyML硬件需支持符号模型。然而,符号模型虽紧凑,其稀疏性与低精度计算需求,与低精度、密集的神经网络模型存在冲突,严重限制了可部署符号模型的规模。本文提出一套软硬件协同框架,实现基于可处理概率电路(PCs)的完整NSAI推理。该框架:(1)训练特定类别的硬件友好型确定性PC,适用于符号任务;(2)采用第n次根压缩技术,将PC压缩至可在TinyML硬件上运行,同时保持最小精度损失;(3)完成整个NSAI模型在嵌入式硬件上的部署。相较于无需压缩的64位精度基准,本方案在FPGA上实现高达82.3%的寄存器使用减少、52.6%的逻辑单元节省和18.0%的Flash占用降低,且在ESP32微控制器上平均推理速度提升4.67倍。
原文摘要 · Abstract (English)
Neurosymbolic AI (NSAI) has recently emerged to mitigate limitations associated with deep learning (DL) models, e.g. quantifying their uncertainty or reason with explicit rules. Hence, TinyML hardware will need to support these symbolic models to bring NSAI to embedded scenarios. Yet, although symbolic models are typically compact, their sparsity and computation resolution contrasts with low-resolution and dense neuro models, which is a challenge on resource-constrained TinyML hardware severely limiting the size of symbolic models that can be computed. In this work, we remove this bottleneck leveraging a tight hardware/software integration to present a complete framework to compute NSAI with TinyML hardware. We focus on symbolic models realized with tractable probabilistic circuits (PCs), a popular subclass of probabilistic models for hardware integration. This framework: (1) trains a specific class of hardware-efficient \emph{deterministic} PCs, chosen for the symbolic task; (2) \emph{compresses} this PC until it can be computed on TinyML hardware with minimal accuracy degradation, using our $n^{th}$-root compression technique, and (3) \emph{deploys} the complete NSAI model on TinyML hardware. Compared to a 64b precision baseline necessary for the PC without compression, our workflow leads to significant hardware reduction on FPGA (up to 82.3\% in FF, 52.6\% in LUTs, and 18.0\% in Flash usage) and an average inference speedup of 4.67x on ESP32 microcontroller.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。