用Transformer预测网络性能，无需训练即可发现新架构
TART: Token-based Architecture Transformer for Neural Network Performance Prediction
- 将网络结构编码为令牌，用Transformer建模架构与性能关系
- 在DeepNets-1M数据集上达到当前最佳预测精度
- 适合自动化设计新网络的研究者和算法工程师
在神经网络架构设计领域，高性能主要依赖研究人员的主观经验。尽管神经架构搜索（NAS）有望实现自动化，但现有方法仍需人工扩展搜索空间，且无法生成新架构。本文探索Transformer在理解神经网络结构及其性能方面的潜力，旨在为利用Transformer生成新网络奠定基础。提出基于令牌的架构Transformer（TART），可在不训练候选网络的情况下预测性能。TART在无边信息条件下于DeepNets-1M数据集上达到当前最优性能，表明Transformer具备辅助发现新型高绩效网络的潜力。
原文摘要 · Abstract (English)
In the realm of neural architecture design, achieving high performance is largely reliant on the manual expertise of researchers. Despite the emergence of Neural Architecture Search (NAS) as a promising technique for automating this process, current NAS methods still require human input to expand the search space and cannot generate new architectures. This paper explores the potential of Transformers in comprehending neural architectures and their performance, with the objective of establishing the foundation for utilizing Transformers to generate novel networks. We propose the Token-based Architecture Transformer (TART), which predicts neural network performance without the need to train candidate networks. TART attains state-of-the-art performance on the DeepNets-1M dataset for performance prediction tasks without edge information, indicating the potential of Transformers to aid in discovering novel and high-performing neural architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。