用两个互补小CNN降低推理能耗,实测节能超85%。
Reducing Inference Energy Consumption Using Dual Complementary CNNs
- 双小CNN协作,一个低置信时由另一个补救,避免大模型开销。
- 在Jetson Nano上实测能耗降低85.8%,准确率仍高。
- 加入记忆模块缓存重复输入结果,进一步省电,适合边缘设备。
卷积神经网络(CNN)的能效已成为研究热点,尽管已有剪枝、量化和硬件优化等方法取得进展,但如何在设备端实现兼顾性能与能效的AI方案仍是挑战。本文提出一种新方法:通过两个小型互补的CNN协同工作,相互弥补预测中的薄弱环节。当第一个CNN的预测置信度较低时,触发第二个CNN以提升置信度。该双CNN结构相比单个大型深层CNN显著降低推理能耗。此外,我们引入一个记忆组件,对相同输入保留先前分类结果,避免重复调用CNN,进一步节省能源。在Jetson Nano平台上,针对每样本重复一次的修改数据集进行实验,最大能耗降低达85.8%。结果表明,采用互补CNN对与记忆机制可在保持高准确率的同时高效降低推理能耗。
原文摘要 · Abstract (English)
Energy efficiency of Convolutional Neural Networks (CNNs) has become an important area of research, with various strategies being developed to minimize the power consumption of these models. Previous efforts, including techniques like model pruning, quantization, and hardware optimization, have made significant strides in this direction. However, there remains a need for more effective on device AI solutions that balance energy efficiency with model performance. In this paper, we propose a novel approach to reduce the energy requirements of inference of CNNs. Our methodology employs two small Complementary CNNs that collaborate with each other by covering each other's "weaknesses" in predictions. If the confidence for a prediction of the first CNN is considered low, the second CNN is invoked with the aim of producing a higher confidence prediction. This dual-CNN setup significantly reduces energy consumption compared to using a single large deep CNN. Additionally, we propose a memory component that retains previous classifications for identical inputs, bypassing the need to re-invoke the CNNs for the same input, further saving energy. Our experiments on a Jetson Nano computer demonstrate an energy reduction of up to 85.8% achieved on modified datasets where each sample was duplicated once. These findings indicate that leveraging a complementary CNN pair along with a memory component effectively reduces inference energy while maintaining high accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。