arXiv:2501.09954cs.LGcs.AI2025-01中稿 · DATE 2025被引 8

用AI预测最优硬件设计,比传统方法快15%且更准。

AIRCHITECT v2: Learning the Hardware Accelerator Design Space through Unified Representations

  • 用对比学习统一编码复杂设计空间,生成一致中间表示。
  • 提出融合分类与回归的统一表示,精准定位最佳硬件配置。
  • 在10万级真实模型上验证,对新模型也能提速1.7倍。

设计空间探索(DSE)在定制硬件架构中至关重要,尤其对于人工智能等新兴应用,需要高度优化的专用设计。随着深度神经网络(DNNs)和先进基础模型(FMs)的发展,DNN加速器的设计空间呈指数级扩张,且高度非均匀、非凸,难以有效导航与优化。传统DSE依赖搜索方法,需反复采样,耗时长且常无法收敛到全局最优。此前的AIrchitect v1首次将DSE转化为常数时间分类问题。本文提出AIrchitect v2,一种更准确、泛化能力更强的学习型DSE方法,适用于大规模设计空间。我们设计了一个编码器-解码器变换器模型:(a) 通过对比学习将复杂设计空间编码为统一的中间表示;(b) 利用新型统一表示,融合分类与回归优势,在不牺牲精度的前提下高效探索大范围设计空间。在10^5个真实DNN工作负载上的实验表明,AIrchitect v2平均比现有技术提升15%以识别最优设计点。进一步在未见过的模型工作负载(如大语言模型)上测试,所选硬件架构实现推理延迟降低1.7倍。

原文摘要 · Abstract (English)

Design space exploration (DSE) plays a crucial role in enabling custom hardware architectures, particularly for emerging applications like AI, where optimized and specialized designs are essential. With the growing complexity of deep neural networks (DNNs) and the introduction of advanced foundational models (FMs), the design space for DNN accelerators is expanding at an exponential rate. Additionally, this space is highly non-uniform and non-convex, making it increasingly difficult to navigate and optimize. Traditional DSE techniques rely on search-based methods, which involve iterative sampling of the design space to find the optimal solution. However, this process is both time-consuming and often fails to converge to the global optima for such design spaces. Recently, AIrchitect v1, the first attempt to address the limitations of search-based techniques, transformed DSE into a constant-time classification problem using recommendation networks. In this work, we propose AIrchitect v2, a more accurate and generalizable learning-based DSE technique applicable to large-scale design spaces that overcomes the shortcomings of earlier approaches. Specifically, we devise an encoder-decoder transformer model that (a) encodes the complex design space into a uniform intermediate representation using contrastive learning and (b) leverages a novel unified representation blending the advantages of classification and regression to effectively explore the large DSE space without sacrificing accuracy. Experimental results evaluated on 10^5 real DNN workloads demonstrate that, on average, AIrchitect v2 outperforms existing techniques by 15% in identifying optimal design points. Furthermore, to demonstrate the generalizability of our method, we evaluate performance on unseen model workloads (LLMs) and attain a 1.7x improvement in inference latency on the identified hardware architecture.

硬件设计AI加速TransformerDSE

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。