arXiv:2410.07762cs.LG2024-10

动态调整神经网络计算精度,实现能效与性能的灵活平衡。

QoS-Nets: Adaptive Approximate Neural Network Inference

  • 通过可重构近似乘法器动态分配计算资源。
  • 三种工作点下功耗降低15.3%~42.8%,精度损失仅0.3~2.33个百分点。
  • 参数增量仅2.75%,适合移动端实时自适应部署。

为在运行时动态调节神经网络应用的算力消耗,本文提出灵活复用近似乘法器进行层计算。设计搜索算法从大规模候选集中选择指定数量的近似乘法器,并支持重训练以最大化任务性能。与以往方法不同,本方法可输出多个非固定分配方案,使系统能根据环境变化逐步调节服务质量(QoS),通过运行时重新分配乘法器实例实现。为此提出一种微调策略,各工作点共享大部分参数,仅需少量额外参数(每点新增2.75%)。在MobileNetV2上验证:选用4个近似乘法器实现3种工作点,乘法功耗降低15.3%~42.8%,Top-5准确率损失0.3~2.33个百分点,模型参数总量仅增加2.75%。

原文摘要 · Abstract (English)

In order to vary the arithmetic resource consumption of neural network applications at runtime, this work proposes the flexible reuse of approximate multipliers for neural network layer computations. We introduce a search algorithm that chooses an appropriate subset of approximate multipliers of a user-defined size from a larger search space and enables retraining to maximize task performance. Unlike previous work, our approach can output more than a single, static assignment of approximate multiplier instances to layers. These different operating points allow a system to gradually adapt its Quality of Service (QoS) to changing environmental conditions by increasing or decreasing its accuracy and resource consumption. QoS-Nets achieves this by reassigning the selected approximate multiplier instances to layers at runtime. To combine multiple operating points with the use of retraining, we propose a fine-tuning scheme that shares the majority of parameters between operating points, with only a small amount of additional parameters required per operating point. In our evaluation on MobileNetV2, QoS-Nets is used to select four approximate multiplier instances for three different operating points. These operating points result in power savings for multiplications between 15.3% and 42.8% at a Top-5 accuracy loss between 0.3 and 2.33 percentage points. Through our fine-tuning scheme, all three operating points only increase the model's parameter count by only 2.75%.

近似计算动态推理能效优化轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。