通过估算模型部分状态，大幅降低神经网络推理计算量
SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model
- 利用时间序列连续性，跳过部分层的完整计算
- 压缩生成通用中间状态，推理速度提升显著
- 适合资源受限设备上的实时神经网络应用
消费电子曾遵循摩尔定律的微型化趋势。尽管微控制器（MCU）处理能力不断提升，但小型家电中的MCU仍难以运行中等规模的前沿人工神经网络（ANN），尤其是在对时延敏感的场景下。本文提出一种名为散射在线推理（Scattered Online Inference, SOI）的新方法，旨在降低ANN的计算复杂度。SOI利用时间序列数据及模型预测的连续性与季节性特征，实现对深层输出的外推，从而提升处理速度。通过压缩技术，生成更通用的神经网络内部部分状态，避免每次推理都重新计算完整模型，显著减少计算开销。
原文摘要 · Abstract (English)
Consumer electronics used to follow the miniaturization trend described by Moore's Law. Despite increased processing power in Microcontroller Units (MCUs), MCUs used in the smallest appliances are still not capable of running even moderately big, state-of-the-art artificial neural networks (ANNs) especially in time-sensitive scenarios. In this work, we present a novel method called Scattered Online Inference (SOI) that aims to reduce the computational complexity of ANNs. SOI leverages the continuity and seasonality of time-series data and model predictions, enabling extrapolation for processing speed improvements, particularly in deeper layers. By applying compression, SOI generates more general inner partial states of ANN, allowing skipping full model recalculation at each inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。