用图结构统一表示网络架构,让大模型更精准地演化神经网络。
GraphIR: Architecture-Level Search States for LLM-Guided Neural Architecture Evolution

- 构建GraphIR图结构,显式表达张量流动、可编辑模块和接口约束。
- 在120题基准上验证,能准确追踪依赖关系并识别接口风险。
- 集成到OpenEvolve后,搜索性能最优且效率高,适合自动化设计场景。
大型语言模型(LLMs)可直接在可执行的神经网络代码上进行神经架构搜索(NAS)。然而,代码级灵活性无法提供有效的架构状态以支持高效变异:LLM必须从实现细节中推断张量依赖、可编辑组件和兼容性约束。为解决这一表征不匹配问题,我们提出GraphIR,一种面向变异的架构感知中间表示,它在可执行程序基础上补充了对齐变异的候选状态。GraphIR通过三个互补视角组织每个候选:计算骨架(描述张量流动)、变异表面(暴露可编辑模块与操作)和有效性包络(捕获接口契约、传播形状及下游依赖)。为评估方法,我们构建了包含120个问题的NAS-Dependency基准,涵盖六个互补的依赖推理维度。诊断结果表明,GraphIR在识别确切生产者、追踪依赖传播以及诊断接口与故障风险方面表现尤为出色。在包括CLRS在内的六个下游基准上,GraphIR实现了最佳整体搜索性能,同时保持相近模型规模,并在集成至OpenEvolve时展现出有利的端到端NAS效率。结果表明,面向变异的架构状态为可执行神经程序与大模型驱动的架构演化之间提供了有效接口。
原文摘要 · Abstract (English)
Large language models (LLMs) enable neural architecture search (NAS) directly over executable neural network programs. However, code-level flexibility does not provide the architecture state needed for effective mutation: LLMs must infer tensor dependencies, editable components, and compatibility constraints from implementation details. To address this representation mismatch, we propose GraphIR, an architecture-aware intermediate representation that supplements executable programs with a mutation-aligned candidate state. GraphIR organizes each candidate through three complementary views: a computation skeleton describing tensor flow, a mutation surface exposing editable modules and operations, and a validity envelope capturing interface contracts, propagated shapes, and downstream dependencies. To evaluate our method, we construct NAS-Dependency, a 120-question benchmark covering six complementary dependency-reasoning dimensions. The diagnostic shows that GraphIR is particularly effective at identifying exact producer occurrences, tracing dependency propagation, and diagnosing interface and failure risks. Across six downstream benchmarks including CLRS, GraphIR achieves the best overall search performance while maintaining comparable model size and favorable end-to-end NAS efficiency when integrated into OpenEvolve. These results show that a mutation-oriented architecture state provides an effective interface between executable neural programs and LLM-guided architecture evolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。