通过扩展数据通道提升小模型推理准确率,不增加延迟。
DEX: Data Channel Extension for Efficient CNN Inference on Tiny AI Accelerators
- 用分块均匀采样和通道堆叠,将原图空间信息融入输入通道。
- 在四款模型、四组数据上平均提升3.5个百分点准确率,延迟不变。
- 适合资源受限的嵌入式设备,尤其关注精度与延迟平衡的场景。
TinyML致力于在小型设备上运行机器学习模型,因其增强的隐私性、低延迟和低成本而日益受到青睐。近期,微型AI加速器的出现通过显著提升硬件算力,彻底改变了TinyML领域。这些加速器配备多个并行处理器和专用的每处理器内存实例,相较于传统微控制器单元(MCU)性能大幅提升。然而,其有限的数据内存常导致输入图像需下采样,进而引发准确率下降。为应对这一挑战,我们提出数据通道扩展(DEX),一种在微型AI加速器上高效执行CNN的新方法。DEX通过分块均匀采样和通道堆叠,将原始图像的额外空间信息融入输入图像,有效扩展输入通道中的数据。利用未充分利用的处理器和数据内存进行通道扩展,使并行执行成为可能,且不增加推理延迟。我们在四款模型和四组数据集上对微型AI加速器的评估表明,该方法平均提升准确率3.5个百分点,同时保持推理延迟不变。源代码可在https://github.com/Nokia-Bell-Labs/data-channel-extension获取。
原文摘要 · Abstract (English)
Tiny machine learning (TinyML) aims to run ML models on small devices and is increasingly favored for its enhanced privacy, reduced latency, and low cost. Recently, the advent of tiny AI accelerators has revolutionized the TinyML field by significantly enhancing hardware processing power. These accelerators, equipped with multiple parallel processors and dedicated per-processor memory instances, offer substantial performance improvements over traditional microcontroller units (MCUs). However, their limited data memory often necessitates downsampling input images, resulting in accuracy degradation. To address this challenge, we propose Data channel EXtension (DEX), a novel approach for efficient CNN execution on tiny AI accelerators. DEX incorporates additional spatial information from original images into input images through patch-wise even sampling and channel-wise stacking, effectively extending data across input channels. By leveraging underutilized processors and data memory for channel extension, DEX facilitates parallel execution without increasing inference latency. Our evaluation with four models and four datasets on tiny AI accelerators demonstrates that this simple idea improves accuracy on average by 3.5%p while keeping the inference latency the same on the AI accelerator. The source code is available at https://github.com/Nokia-Bell-Labs/data-channel-extension.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。