通过协同分析与精确模拟,高效优化安全推理加速器的模型映射。
Obsidian: Cooperative State-Space Exploration for Performant Inference on Secure ML Accelerators
- 结合分析模型与周期精确仿真,分阶段探索加速器状态空间。
- 云边端部署下延迟降低最高达20.5%,能耗改善最高达24%。
- 适合需要兼顾安全与性能的ML加速器优化场景。
面向机器学习加速器的可信执行环境(TEEs)在保障安全与提升效率方面至关重要。通过状态空间探索优化工作负载可显著提升性能与能效,但受限于庞大的搜索空间,现有方法要么依赖快速解析模型而忽略硬件细节,要么使用周期精确模型却因运行时间过长难以覆盖大范围。本文提出Obsidian框架,通过协同使用分析模型与周期精确模型,实现对安全型机器学习加速器最优模型映射的搜索。其核心包含:(1) 融入安全硬件影响的分析模型,在大规模映射空间中快速生成最优候选;(2) 基于周期精确模型的编译器剖析步骤,捕捉运行时瓶颈以进一步优化执行延迟、能耗与资源利用率。实验对比采用GuardNN [33] 和 Sesame [11] 的先进安全方案作为基线,分析模型使云端推理延迟降低20.5%,边缘端降低8.4%,能耗分别提升24%和19%;周期精确模型进一步将云端延迟再降9.1%,边缘端降12.2%,能耗分别再增13.8%和13.1%。
原文摘要 · Abstract (English)
Trusted execution environments (TEEs) for machine learning accelerators are indispensable in secure and efficient ML inference. Optimizing workloads through state-space exploration for the accelerator architectures improves performance and energy consumption. However, such explorations are expensive and slow due to the large search space. Current research has to use fast analytical models that forego critical hardware details and cross-layer opportunities unique to the hardware security primitives. While cycle-accurate models can theoretically reach better designs, their high runtime cost restricts them to a smaller state space. We present Obsidian, an optimization framework for finding the optimal mapping from ML kernels to a secure ML accelerator. Obsidian addresses the above challenge by exploring the state space using analytical and cycle-accurate models cooperatively. The two main exploration components include: (1) A secure accelerator analytical model, that includes the effect of secure hardware while traversing the large mapping state space and produce the best m model mappings; (2) A compiler profiling step on a cycle-accurate model, that captures runtime bottlenecks to further improve execution runtime, energy and resource utilization and find the optimal model mapping. We compare our results to a baseline secure accelerator, comprising of the state-of-the-art security schemes obtained from guardnn [ 33 ] and sesame [11]. The analytical model reduces the inference latency by 20.5% for a cloud and 8.4% for an edge deployment with an energy improvement of 24% and 19% respectively. The cycle-accurate model, further reduces the latency by 9.1% for a cloud and 12.2% for an edge with an energy improvement of 13.8% and 13.1%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。