arXiv:2510.01472cs.LG2025-10被引 2

用大模型加速神经网络搜索,兼顾精度与速度,节省大量计算资源。

LLM-NAS: LLM-driven Hardware-Aware Neural Architecture Search

  • 按复杂度划分搜索空间,避免重复设计,提升多样性。
  • 大模型结合知识进化,每轮优化提示与架构,效率更高。
  • 零成本预测替代训练,搜索时间从天级降至分钟级。

硬件感知神经网络架构搜索(HW-NAS)需在设备约束下联合优化准确率与延迟。传统基于超网络的方法每数据集需数个GPU天。大语言模型(LLM)驱动方法可避免训练大型超网络并快速反馈,但存在探索偏差:模型反复提出有限搜索空间内的设计,难以发现跨不同延迟范围的架构。为此,我们提出LLM-NAS:一种低开销、高效率的LLM驱动架构搜索方法。其包含三个核心组件:1)复杂度驱动的分区引擎,按复杂度划分搜索空间以增强多样性并缓解探索偏差;2)基于LLM的架构提示协同进化算子,通过前一轮结果更新设计经验库,再以融合该知识的提示引导架构进化,提示与架构同步优化,减少随机尝试;3)零成本预测器,避免从头训练大量候选架构。实验表明,在HW-NAS-Bench上,LLM-NAS实现更高的超体积(HV)、更低的逆差距(IGD),在相近准确率下延迟降低最高达54%,搜索成本从数天降至分钟级。

原文摘要 · Abstract (English)

Hardware-Aware Neural Architecture Search (HW-NAS) requires joint optimization of accuracy and latency under device constraints. Traditional supernet-based methods require multiple GPU days per dataset. Large Language Model (LLM)-driven approaches avoid training a large supernet and can provide quick feedback, but we observe an exploration bias: the LLM repeatedly proposes neural network designs within limited search space and fails to discover architectures across different latency ranges in the entire search space. To address this issue, we propose LLM-NAS: an LLM-driven Neural Architecture Search that can generate neural networks with high accuracy and low latency with reduced search cost. Our proposed LLM-NAS has three key components: 1) a complexity-driven partitioning engine that divides the search space by complexity to enforce diversity and mitigate exploration bias; 2) an LLM-powered architecture prompt co-evolution operator, in which the LLM first updates a knowledge base of design heuristics based on results from the previous round, then performs a guided evolution algorithm on architectures with prompts that incorporate this knowledge base. Prompts and designs improve together across rounds which avoids random guesswork and improve efficiency; 3) a zero-cost predictor to avoid training a large number of candidates from scratch. Experimental results show that on HW-NAS-Bench, LLM-NAS can achieve overall higher HV, lower IGD, and up to 54% lower latency than baselines at similar accuracy. Meanwhile, the search cost drops from days to minutes compared with traditional supernet baselines.

神经网络搜索大模型硬件感知高效搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。