用大模型直接生成神经网络参数，实现零训练的模型构建
NeuroGen: Neural Network Parameter Generation via Large Language Models
- 分两阶段：先预训练大模型理解参数空间，再通过任务提示微调
- 在多个数据集上生成的参数可直接使用，准确率接近传统方法
- 适合快速原型设计、小样本场景下模型生成的科研与工程人员
获取神经网络（NN）参数是机器学习自诞生以来最重要的问题之一。传统方法如反向传播和前向优化依赖迭代数据拟合逐步优化参数。本文探索一种新方向：通过大语言模型生成神经网络参数。提出NeuroGen，一种通用且易于实现的两阶段方法，根据数据、任务和网络架构描述生成参数。第一阶段为参数参考知识注入，将大模型在神经网络检查点上预训练，建立对参数空间的基础理解；第二阶段为上下文增强指令微调，通过丰富且任务感知的提示使大模型适应具体任务。实验表明，NeuroGen能有效生成可用的神经网络参数。研究结果证明了基于大模型的神经网络参数生成可行性，暗示了大模型与轻量级神经网络协同共存的新范式。
原文摘要 · Abstract (English)
Acquiring the parameters of neural networks (NNs) has been one of the most important problems in machine learning since the inception of NNs. Traditional approaches, such as backpropagation and forward-only optimization, acquire parameters via iterative data fitting to gradually optimize them. This paper aims to explore the feasibility of a new direction: acquiring NN parameters via large language model generation. We propose NeuroGen, a generalized and easy-to-implement two-stage approach for NN parameter generation conditioned on descriptions of the data, task, and network architecture. Stage one is Parameter Reference Knowledge Injection, where LLMs are pretrained on NN checkpoints to build foundational understanding of parameter space, whereas stage two is Context-Enhanced Instruction Tuning, enabling LLMs to adapt to specific tasks through enriched, task-aware prompts. Experimental results demonstrate that NeuroGen effectively generates usable NN parameters. Our findings highlight the feasibility of LLM-based NN parameter generation and suggest a promising new paradigm where LLMs and lightweight NNs can coexist synergistically
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。