arXiv:2411.02471cs.LGcs.AI2024-11被引 2

动态调整神经网络推理以适应能量波动,提升设备续航与精度。

Energy-Aware Dynamic Neural Inference

  • 根据能量状态动态切换模型大小或提前输出结果
  • 能量充足时准确率比传统方法高约5%
  • 适合低功耗、自供能的边缘设备应用

随着智能应用对边缘外设备的需求增长,可持续运行成为关键挑战。由于环境能量来源具有随机性,常导致能量捕获不足,无法满足推理需求,造成性能下降。为此,本文设计一种集成能量采集器和有限储能的本地自适应推理系统,通过多模型选择(MMS)或早期退出(EE)机制按需降低运行成本。模型选择或退出点由当前能量状态动态决定,并引入预测置信度优化决策。我们推导出具备理论保证的置信度感知与无感知控制策略;在多出口网络中,采用轻量级强化学习实现逐级决策。实验表明,当环境能量获取速率上升时,置信度感知的控制方案相比仅考虑能量的方案,准确率提升约5%;在储能容量相对较小的情况下,增量式决策进一步提高精度。

原文摘要 · Abstract (English)

The growing demand for intelligent applications beyond the network edge, coupled with the need for sustainable operation, are driving the seamless integration of deep learning (DL) algorithms into energy-limited, and even energy-harvesting end-devices. However, the stochastic nature of ambient energy sources often results in insufficient harvesting rates, failing to meet the energy requirements for inference and causing significant performance degradation in energy-agnostic systems. To address this problem, we consider an on-device adaptive inference system equipped with an energy-harvester and finite-capacity energy storage. We then allow the device to reduce the run-time execution cost on-demand, by either switching between differently-sized neural networks, referred to as multi-model selection (MMS), or by enabling earlier predictions at intermediate layers, called early exiting (EE). The model to be employed, or the exit point is then dynamically chosen based on the energy storage and harvesting process states. We also study the efficacy of integrating the prediction confidence into the decision-making process. We derive a principled policy with theoretical guarantees for confidence-aware and -agnostic controllers. Moreover, in multi-exit networks, we study the advantages of taking decisions incrementally, exit-by-exit, by designing a lightweight reinforcement learning-based controller. Experimental results show that, as the rate of the ambient energy increases, energy- and confidence-aware control schemes show approximately 5% improvement in accuracy compared to their energy-aware confidence-agnostic counterparts. Incremental approaches achieve even higher accuracy, particularly when the energy storage capacity is limited relative to the energy consumption of the inference model.

边缘计算节能推理动态调度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。