用真实硬件反馈优化边缘AI模型，速度提升8.8倍且精度更高
Optimizing edge AI models on HPC systems with the edge in the loop
- 通过硬件感知的神经架构搜索，在德国超算与比利时边缘设备间协同优化
- 在增材制造数据集上实现推理速度提升8.8倍，精度提升1.35倍
- 适合需低延迟高精度的工业边缘计算场景
部署于边缘设备的人工智能模型（如增材制造中的质量控制）通常体积小，需在短时间内完成高精度推理。现有方法多从大模型出发，通过结构剪枝、知识蒸馏或量化压缩。本文提出一种硬件感知神经架构搜索流程，将位于比利时的边缘设备与德国高性能计算系统联动，快速训练候选模型并实时测量目标硬件延迟。该方法在公开的RAISE-LPBF数据集上验证，相较人工设计基线，推理速度提升约8.8倍，同时模型性能提升约1.35倍。
原文摘要 · Abstract (English)
Artificial intelligence and machine learning models deployed on edge devices, e.g., for quality control in Additive Manufacturing (AM), are frequently small in size. Such models usually have to deliver highly accurate results within a short time frame. Methods that are commonly employed in literature start out with larger trained models and try to reduce their memory and latency footprint by structural pruning, knowledge distillation, or quantization. It is, however, also possible to leverage hardware-aware Neural Architecture Search (NAS), an approach that seeks to systematically explore the architecture space to find optimized configurations. In this study, a hardware-aware NAS workflow is introduced that couples an edge device located in Belgium with a powerful High-Performance Computing system in Germany, to train possible architecture candidates as fast as possible while performing real-time latency measurements on the target hardware. The approach is verified on a use case in the AM domain, based on the open RAISE-LPBF dataset, achieving ~8.8 times faster inference speed while simultaneously enhancing model quality by a factor of ~1.35, compared to a human-designed baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。