用自然语言生成神经网络权重，让模型自动适应新任务。
Text2Weight: Bridging Natural Language and Neural Network Weight Spaces
- 用扩散Transformer架构，将文本描述转为任务专用权重
- 在三个数据集上生成的权重能适配未见任务，性能超越传统初始化方法
- 适合对自动化模型设计、文本驱动神经网络感兴趣的开发者
我们提出T2W，一种基于扩散Transformer的框架，可依据自然语言描述生成特定任务的神经网络权重。该方法将网络参数分块处理，通过先验注意力机制融合CLIP文本嵌入，并采用权重空间增强的对抗训练提升泛化能力。在Cifar100、Caltech256和TinyImageNet上的实验表明，T2W能为未见过的任务生成高质量权重，优于基于优化的初始化方法，并支持权重增强与文本引导的模型融合等新应用。研究实现了文本语义与权重空间动态的桥梁构建，配套开源了包含文本-权重对的数据集，推动生成式模型在神经网络参数合成中的实际应用。代码已公开于Github。
原文摘要 · Abstract (English)
How far are we really from automatically generating neural networks? While neural network weight generation shows promise, current approaches struggle with generalization to unseen tasks and practical application exploration. To address this, we propose T2W, a diffusion transformer framework that generates task-specific weights conditioned on natural language descriptions. T2W hierarchically processes network parameters into uniform blocks, integrates text embeddings from CLIP via a prior attention mechanism, and employs adversarial training with weight-space augmentation to enhance generalization. Experiments on Cifar100, Caltech256, and TinyImageNet demonstrate T2W's ability to produce high-quality weights for unseen tasks, outperforming optimization-based initialization and enabling novel applications such as weight enhancement and text-guided model fusion. Our work bridges textual semantics with weight-space dynamics, supported by an open-source dataset of text-weight pairs, advancing the practicality of generative models in neural network parameter synthesis. Our code is available on Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。