用分步引导的编辑策略，让大模型更精准地优化神经网络结构。
Structured Progressive Knowledge Activation for LLM-Driven Neural Architecture Search
- 通过指定要修改的功能模块来控制代码改动，减少副作用。
- 在CLRS-DFS数据集上实现28.1倍采样效率提升，外部泛化准确率提高22.9%。
- 适合需要高效探索新架构的研究者，尤其关注大模型辅助设计。
本文聚焦神经网络架构搜索（NAS）中的关键挑战：在高成本评估下，如何整合已有架构知识并探索新设计。大语言模型（LLMs）有望作为NAS助手，将丰富的架构与编码先验转化为可执行的代码修改。然而实践中，看似局部的修改常引发非局部的行为与性能变化，这是因为单次修改可能意外耦合多个相互作用的功能因子，我们称之为功能纠缠。为应对这种纠缠，本文提出结构化渐进式知识激活（SPARK），通过显式选择需修改的功能因子，并基于该因子条件化编辑操作，从而降低纠缠带来的副作用，实现更精准、可靠的架构调整。在CLRS-DFS数据集上，SPARK实现28.1倍的样本效率提升，并带来22.9%的外部泛化准确率相对改进。代码已公开于https://github.com/AIM-ResearchLab/SPARK。
原文摘要 · Abstract (English)
This paper focuses on a key challenge in Neural Architecture Search (NAS): integrating established architectural knowledge while exploring new designs under expensive evaluations. Large language models (LLMs) are a promising assistant for NAS because they can translate rich architectural and coding priors into executable code edits. However, in practice, seemingly local revisions often propagate into non-local behavioral and performance shifts because a single edit can inadvertently couple multiple interacting functional factors, a phenomenon we refer to as functional entanglement. To make LLM knowledge usable under such entanglement, we propose Structured Progressive Knowledge Activation (SPARK), which activates relevant priors by explicitly selecting the functional factor to modify and conditioning the edit on that factor. This factor-conditioned editing reduces entangled side effects and yields more targeted, reliable architecture modifications. On CLRS-DFS, SPARK achieves a 28.1x sample-efficient architecture evolution speedup and yields a 22.9\% relative improvement in OOD accuracy. Our code is available at https://github.com/AIM-ResearchLab/SPARK.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。