arXiv:2509.18151cs.LGcs.AI2025-09被引 1

用超网络提升神经架构搜索预测器的表示能力,少用数据也能更准。

HyperNAS: Enhancing Architecture Representation for NAS Predictor via Hypernetwork

  • 设计全局编码+共享超网络,更好捕捉架构间复杂关系
  • 在少量样本下达到97.6%(CIFAR-10)和82.4%(ImageNet)准确率
  • 适合资源有限的少样本场景,尤其适用于ViT等新型架构

神经架构搜索(NAS)因耗时的性能评估而进展缓慢。现有预测器依赖代理数据集上的替代模型进行直接性能预测,但往往泛化能力差,难以捕捉不同架构间的复杂关联。本文提出HyperNAS,一种新型神经预测范式,通过全局编码方案捕获架构的宏观结构信息,并引入共享超网络作为辅助任务,增强对跨架构模式的挖掘。为保障训练稳定,进一步设计动态自适应多任务损失,支持在帕累托前沿上的个性化探索。在五个代表性搜索空间(包括ViTs)上广泛实验表明,HyperNAS在少样本场景下优势显著:例如,在至少减少5.0倍样本量的前提下,取得CIFAR-10上97.60%的top-1准确率和ImageNet上82.4%的top-1准确率,刷新当前最佳表现。

原文摘要 · Abstract (English)

Time-intensive performance evaluations significantly impede progress in Neural Architecture Search (NAS). To address this, neural predictors leverage surrogate models trained on proxy datasets, allowing for direct performance predictions for new architectures. However, these predictors often exhibit poor generalization due to their limited ability to capture intricate relationships among various architectures. In this paper, we propose HyperNAS, a novel neural predictor paradigm for enhancing architecture representation learning. HyperNAS consists of two primary components: a global encoding scheme and a shared hypernetwork. The global encoding scheme is devised to capture the comprehensive macro-structure information, while the shared hypernetwork serves as an auxiliary task to enhance the investigation of inter-architecture patterns. To ensure training stability, we further develop a dynamic adaptive multi-task loss to facilitate personalized exploration on the Pareto front. Extensive experiments across five representative search spaces, including ViTs, demonstrate the advantages of HyperNAS, particularly in few-shot scenarios. For instance, HyperNAS strikes new state-of-the-art results, with 97.60\% top-1 accuracy on CIFAR-10 and 82.4\% top-1 accuracy on ImageNet, using at least 5.0$\times$ fewer samples.

神经架构搜索预测器少样本超网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。