通过专家投票机制提升早期退出模型的准确率与速度
BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts
- 将多个中间出口视为专家，仅当预测一致时才聚合置信度
- 在COCO图像描述和GLUE语言任务上实现1.5至2.1倍加速
- 适合追求低延迟且需保持高精度的实时AI应用
早期退出（Early Exit, EE）技术已成为降低深度神经网络（DNN）推理延迟的有效手段。其性能优劣关键取决于退出决策标准。本文提出一种新方法——BEEM，将各中间出口分类器视为专家，并仅在相邻专家预测一致时聚合其置信度，从而捕捉集成效应。样本在累积置信度超过阈值时退出。该阈值基于中间层错误率设定，旨在超越传统DNN推理表现。在COCO图像描述任务及GLUE系列语言任务上的实验表明，本方法显著提升现有EE方法性能，加速比达1.5倍至2.1倍。相较于最终层输出，在更难的图像描述任务中准确率相当，在较易的语言任务中甚至更高。代码已开源。
原文摘要 · Abstract (English)
Early Exit (EE) techniques have emerged as a means to reduce inference latency in Deep Neural Networks (DNNs). The latency improvement and accuracy in these techniques crucially depend on the criteria used to make exit decisions. We propose a new decision criterion where exit classifiers are treated as experts BEEM and aggregate their confidence scores. The confidence scores are aggregated only if neighbouring experts are consistent in prediction as the samples pass through them, thus capturing their ensemble effect. A sample exits when the aggregated confidence value exceeds a threshold. The threshold is set using the error rates of the intermediate exits aiming to surpass the performance of conventional DNN inference. Experimental results on the COCO dataset for Image captioning and GLUE datasets for various language tasks demonstrate that our method enhances the performance of state-of-the-art EE methods, achieving improvements in speed-up by a factor 1.5x to 2.1x. When compared to the final layer, its accuracy is comparable in harder Image Captioning and improves in the easier language tasks. The source code for this work is publicly available at https://github.com/Div290/BEEM1/tree/main
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。