为基于大模型的神经架构搜索提供收敛理论，解释为何搜索会有效并如何避免失效。
Convergence Theory for Iterative LLM-Based Neural Architecture Search: A Parametric Cross-Entropy Framework with Closed-Form Proxy Reliability
- 将大模型搜索建模为参数化交叉熵优化，用数学证明其迭代过程稳定
- 实验证明搜索能持续提升架构质量，且精英集概率按几何速度收敛
- 提出可计算的代理可靠性公式，指导何时信任搜索结果
大语言模型（LLM）在迭代神经架构搜索（NAS）中日益作为生成器使用，但此类算法缺乏正式的收敛理论。本文将迭代LLM-NAS建模为对可执行程序的参数化交叉熵（CE）方法，并证明了六个结论：（1）在精英架构上迭代微调大模型等价于限制在大模型参数族内的CE更新；（2）各轮次预期架构质量单调不减；（3）精英集概率以几何速率收敛，收敛下界为1-(1−ρ₀)^t；（4）在首阶马尔可夫词元错误模型下，基于delta的生成方式比全代码生成具有更高的有效生成率；（5）MinHash-Jaccard新颖性过滤器可防止模式崩溃；（6）代理可靠性存在闭式表达式ρ_S = (6/π) arcsin(ρ_P(SNR)/2)，由此得出σ²_arch >> σ²_noise是可信代理排序的必要条件。在包含22轮、3个大模型、6个数据集、共3300个生成架构的实验中，定量验证了两个预测，方向性验证了两个，且解释了此前经验发现但未被阐明的代理可靠性天花板现象。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used as generators in iterative neural architecture search (NAS), yet no formal convergence theory exists for this class of algorithms. We model iterative LLM-NAS as a parametric Cross-Entropy (CE) method over executable programs and prove six results: (1) iterative LLM fine-tuning on elite architectures is equivalent to the CE update restricted to the LLM parametric family; (2) expected architecture quality is monotonically non-decreasing across cycles; (3) elite-set probability converges to a fixed point at a geometric rate C_t >= 1-(1-rho_0)^t; (4) delta-based generation achieves a strictly higher valid-generation rate than full-code generation under a first-order Markov token-error model; (5) the MinHash-Jaccard novelty filter prevents mode collapse; (6) proxy reliability admits the closed-form rho_S = (6/pi) arcsin(rho_P(SNR)/2), yielding the practical diagnostic sigma^2_arch >> sigma^2_noise as a necessary condition for trustworthy proxy-based rankings. Testing against a 22-cycle, three-LLM, six-dataset experiment with 3,300 generated architectures confirms two predictions quantitatively, two at direction-of-effect level, and explains the proxy-reliability ceiling effect previously reported empirically but left unexplained.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。