arXiv:2509.05657cs.CLcs.AI2025-09EMNLP被引 8

用统一数字编码让大模型跨领域搜模型,无需调参

LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encoding

  • 用数值编码统一表示神经网络结构,支持跨任务搜索
  • 在图像分类、分割生成等任务上表现媲美专用方法
  • 适合想快速部署通用模型搜索的算法工程师

大语言模型(LLM)为解决复杂优化问题提供了新路径,包括神经网络架构搜索(NAS)。然而,现有基于LLM的NAS方法严重依赖提示工程和领域特定调优,限制了其在多样化任务中的实用性与可扩展性。本文提出LM-Searcher,一种无需领域特化适配即可实现跨域神经架构优化的新框架。核心是NCode——一种用于神经架构的通用数值字符串表示,支持跨域架构编码与搜索。我们将NAS重构为排序任务,通过基于剪枝的子空间采样策略生成指令微调样本,训练LLM从候选池中选出高性能架构。我们构建了一个涵盖多种架构-性能对的高质量数据集,促进鲁棒且可迁移的学习。大量实验表明,LM-Searcher在同域(如图像分类的CNN)和跨域(如分割与生成任务的LoRA配置)任务中均取得竞争力表现,确立了灵活通用的基于大模型的架构搜索新范式。数据集与模型将开源于https://github.com/Ashone3/LM-Searcher。

原文摘要 · Abstract (English)

Recent progress in Large Language Models (LLMs) has opened new avenues for solving complex optimization problems, including Neural Architecture Search (NAS). However, existing LLM-driven NAS approaches rely heavily on prompt engineering and domain-specific tuning, limiting their practicality and scalability across diverse tasks. In this work, we propose LM-Searcher, a novel framework that leverages LLMs for cross-domain neural architecture optimization without the need for extensive domain-specific adaptation. Central to our approach is NCode, a universal numerical string representation for neural architectures, which enables cross-domain architecture encoding and search. We also reformulate the NAS problem as a ranking task, training LLMs to select high-performing architectures from candidate pools using instruction-tuning samples derived from a novel pruning-based subspace sampling strategy. Our curated dataset, encompassing a wide range of architecture-performance pairs, encourages robust and transferable learning. Comprehensive experiments demonstrate that LM-Searcher achieves competitive performance in both in-domain (e.g., CNNs for image classification) and out-of-domain (e.g., LoRA configurations for segmentation and generation) tasks, establishing a new paradigm for flexible and generalizable LLM-based architecture search. The datasets and models will be released at https://github.com/Ashone3/LM-Searcher.

架构搜索大模型应用跨领域

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。