嵌入式设备推理时，乘加操作不能准确反映实际耗时。
Hidden costs for inference with deep network on embedded system devices
- 用乘加操作衡量计算量会忽略张量间额外开销。
- 实测显示不同模型推理时间与乘加数不一致。
- 适合做边缘端模型优化的研究者参考。
本研究评估了多种深度学习模型在嵌入式系统环境下的推理性能。以往工作通常以乘加操作（Multiply-Accumulate）作为衡量模型计算负载的指标。然而，本研究指出该指标在预测嵌入式设备上的推理时间时存在局限性。论文探讨了仅依赖乘加操作所忽视的关键因素。实验在嵌入式设备上使用CIFAR-100数据集进行图像分类任务，对比分析了十种深度模型的推理时间与各自理论乘加操作数。结果表明，在优化深度模型以实现嵌入式系统实时运行时，必须考虑张量间的额外计算开销。
原文摘要 · Abstract (English)
This study evaluates the inference performance of various deep learning models under an embedded system environment. In previous works, Multiply-Accumulate operation is typically used to measure computational load of a deep model. According to this study, however, this metric has a limitation to estimate inference time on embedded devices. This paper poses the question of what aspects are overlooked when expressed in terms of Multiply-Accumulate operations. In experiments, an image classification task is performed on an embedded system device using the CIFAR-100 dataset to compare and analyze the inference times of ten deep models with the theoretically calculated Multiply-Accumulate operations for each model. The results highlight the importance of considering additional computations between tensors when optimizing deep learning models for real-time performing in embedded systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。