优化安卓端AI模型推理，平衡速度与精度损失。
Hardware optimization on Android for inference of AI models
- 测试不同量化方案与GPU/NPU协同调度策略
- 在保持精度损失小于2%前提下提速超3倍
- 适合移动端AI部署开发者参考
人工智能模型在现代移动计算中广泛应用，涵盖虚拟助手到图像处理等场景。优化用户体验需兼顾低延迟与高响应性，挑战在于如何充分利用实时约束和异构硬件架构。本文研究并提出适用于Android系统的AI模型最优执行配置，聚焦目标检测（YOLO系列）与图像分类（ResNet）两类任务。通过评估多种模型量化方案及本地加速器（特别是GPU和NPU）的使用方式，旨在实证确定在最小精度下降与最大推理加速之间最佳平衡点。
原文摘要 · Abstract (English)
The pervasive integration of Artificial Intelligence models into contemporary mobile computing is notable across numerous use cases, from virtual assistants to advanced image processing. Optimizing the mobile user experience involves minimal latency and high responsiveness from deployed AI models with challenges from execution strategies that fully leverage real time constraints to the exploitation of heterogeneous hardware architecture. In this paper, we research and propose the optimal execution configurations for AI models on an Android system, focusing on two critical tasks: object detection (YOLO family) and image classification (ResNet). These configurations evaluate various model quantization schemes and the utilization of on device accelerators, specifically the GPU and NPU. Our core objective is to empirically determine the combination that achieves the best trade-off between minimal accuracy degradation and maximal inference speed-up.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。