用大模型自进化和知识提炼，高效搜索出顶尖神经网络结构。
SEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Models
- 通过迭代优化积累高性能架构，再提炼共性生成新设计。
- 仅需0.05 GPU天即达顶尖性能，效率远超现有方法。
- 无需领域数据，通用性强，适合多任务场景使用。
我们提出SEKI，一种基于大语言模型（LLM）的神经架构搜索（NAS）新方法。受现代LLM链式思维（CoT）启发，SEKI分为两个阶段：自进化与知识提炼。在自进化阶段，由于初始缺乏参考样本，采用迭代精炼机制，依据性能反馈不断优化架构，逐步积累高质量架构库。在知识提炼阶段，LLM分析这些架构中的共性模式，生成新的优化设计。结合两阶段优势，SEKI充分发挥了LLM在NAS中的潜力，且无需任何领域特定数据。实验表明，SEKI在多个数据集和搜索空间上均达到当前最优（SOTA）表现，仅需0.05 GPU天，同时在效率与精度上超越现有方法。此外，其泛化能力突出，在多种任务上均取得接近SOTA的结果。
原文摘要 · Abstract (English)
We introduce SEKI, a novel large language model (LLM)-based neural architecture search (NAS) method. Inspired by the chain-of-thought (CoT) paradigm in modern LLMs, SEKI operates in two key stages: self-evolution and knowledge distillation. In the self-evolution stage, LLMs initially lack sufficient reference examples, so we implement an iterative refinement mechanism that enhances architectures based on performance feedback. Over time, this process accumulates a repository of high-performance architectures. In the knowledge distillation stage, LLMs analyze common patterns among these architectures to generate new, optimized designs. Combining these two stages, SEKI greatly leverages the capacity of LLMs on NAS and without requiring any domain-specific data. Experimental results show that SEKI achieves state-of-the-art (SOTA) performance across various datasets and search spaces while requiring only 0.05 GPU-days, outperforming existing methods in both efficiency and accuracy. Furthermore, SEKI demonstrates strong generalization capabilities, achieving SOTA-competitive results across multiple tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。