arXiv:2603.05904cs.ARcs.AI2026-03被引 1

用大模型分析瓶颈,20步就找到比A100更好的GPU设计。

LUMINA: LLM-Guided GPU Architecture Exploration via Bottleneck Analysis

  • 用大模型自动提炼设计规则,结合瓶颈分析优化探索路径。
  • 在470万种可能中仅用20步找到6个更优方案,效率提升17.5倍。
  • 适合芯片设计、AI系统优化等领域的研究与工程人员参考。

针对现代AI工作负载(如大语言模型推理)的GPU架构探索(DSE)面临设计空间庞大、仿真成本高及多目标权衡(性能、功耗、面积)等挑战。现有自动化DSE方法通常代价过高,或依赖人工构建的复杂关键路径分析。本文提出LUMINA,一个由大模型驱动的GPU架构探索框架,通过从仿真器代码中提取架构知识并进行敏感性分析,自动构建并动态修正DSE规则。核心组件是用于评估和提升大模型在架构优化中三项基础能力的基准测试,确保模型选择与推理一致性。在包含470万种配置的设计空间中,LUMINA仅用20步便高效识别出6个性能与面积优于A100的方案;相比机器学习基线,其探索效率提升17.5倍,设计方案质量(帕累托超体积)提高32.9%,证明了其以极低搜索成本提供高质量设计指导的能力。

原文摘要 · Abstract (English)

GPU design space exploration (DSE) for modern AI workloads, such as Large-Language Model (LLM) inference, is challenging because of GPUs' vast, multi-modal design spaces, high simulation costs, and complex design optimization objectives (e.g. performance, power and area trade-offs). Existing automated DSE methods are often prohibitively expensive, either requiring an excessive number of exploration samples or depending on intricate, manually crafted analyses of interdependent critical paths guided by human heuristics. We present LUMINA, an LLM-driven GPU architecture exploration framework that leverage AI to enhance the DSE efficiency and efficacy for GPUs. LUMINA extracts architectural knowledge from simulator code and performs sensitivity studies to automatically compose DSE rules,which are auto-corrected during exploration. A core component of LUMINA is a DSE Benchmark that comprehensively evaluates and enhances LLMs' capabilities across three fundamental skills required for architecture optimization, which provides a principled and reproducible basis for model selection and ensuring consistent architectural reasoning. In the design space with 4.7 million possible samples, LUMINA identifies 6 designs of better performance and area than an A100 GPU efficiently, using only 20 steps via LLM-assisted bottleneck analysis. In comparison, LUMINA achieves 17.5x higher than design space exploration efficiency, and 32.9% better designs (i.e. Pareto Hypervolume) than Machine-Learning baselines, showcasing its ability to deliver high-quality design guidance with minimal search cost.

GPU设计大模型应用架构探索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。