arXiv:2601.05052cs.LGstat.ML2026-01被引 2

直接在权重空间生成高精度神经网络,无需微调且支持大规模模型。

DeepWeightFlow: Re-Basined Flow Matching for Generating Neural Network Weights

  • 基于流匹配直接生成完整权重,避免部分权重生成限制。
  • 生成的网络无需微调即表现优异,百个模型生成仅需数分钟。
  • 结合重基线和TransFusion提升大模型生成效率,适合迁移学习场景。

构建高效有效的神经网络权重生成模型是当前研究热点,但面临现代神经网络高维权重空间及对称性挑战。此前方法多局限于生成部分权重,或生成后需微调,难以扩展至大型模型(如ResNet、ViT)。本文提出DeepWeightFlow,一种直接在权重空间操作的流匹配模型,可生成多样且高精度的神经网络权重,适用于多种架构、规模与数据模态。生成网络无需微调即可表现良好,并可拓展至大模型。引入Git Re-Basin与TransFusion进行神经网络规范化,有效处理权重排列对称性,提升大模型生成效率。生成的网络在迁移学习中表现卓越,数百个模型可在数分钟内生成,远超扩散模型效率。DeepWeightFlow为高效可扩展的神经网络集合生成开辟新路径。

原文摘要 · Abstract (English)

Building efficient and effective generative models for neural network weights has been a research focus of significant interest that faces challenges posed by the high-dimensional weight spaces of modern neural networks and their symmetries. Several prior generative models are limited to generating partial neural network weights, particularly for larger models, such as ResNet and ViT. Those that do generate complete weights struggle with generation speed or require finetuning of the generated models. In this work, we present DeepWeightFlow, a Flow Matching model that operates directly in weight space to generate diverse and high-accuracy neural network weights for a variety of architectures, neural network sizes, and data modalities. The neural networks generated by DeepWeightFlow do not require fine-tuning to perform well and can scale to large networks. We apply Git Re-Basin and TransFusion for neural network canonicalization in the context of generative weight models to account for the impact of neural network permutation symmetries and to improve generation efficiency for larger model sizes. The generated networks excel at transfer learning, and ensembles of hundreds of neural networks can be generated in minutes, far exceeding the efficiency of diffusion-based methods. DeepWeightFlow models pave the way for more efficient and scalable generation of diverse sets of neural networks.

权重生成流匹配迁移学习高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。