用元学习提升神经网络架构搜索的预测泛化能力
From Regression to Inference: Meta-Learning Predictors for Neural Architecture Search
- 将性能预测转为条件函数推断，通过元学习从少量样本中推断
- 在NAS-Bench-101和201上以有限样本实现最优排名质量
- 适合资源受限场景下的高效架构选择，尤其适用于小样本
基于预测的方法广泛用于神经架构搜索（NAS），通过预测器估计候选架构的性能以指导选择。然而，现有预测器通常在有限样本上通过监督回归训练，导致过拟合且对未见架构泛化能力差。本文提出一种根本性新范式，将性能预测建模为使用具备元学习能力的卷积神经过程（ConvNP）的条件函数推断问题。不同于固定映射拟合有限样本，本方法通过在一组合成任务中进行上下文-目标划分训练，元学习从部分观测中推断性能，显式优化数据稀缺下的泛化能力，并使训练过程与NAS部署设置对齐。我们进一步设计了适用于单元结构的简单有效元特征，并在NAS-Bench-101和NAS-Bench-201上进行了评估。大量实验表明，该方法在有限样本下持续提升前K名架构排名质量，达到当前最优架构选择效果。
原文摘要 · Abstract (English)
Prediction-based approaches are widely used in neural architecture search (NAS), where a predictor estimates the performance of candidate architectures to guide selection. However, existing predictors are typically trained via supervised regression on limited samples, leading to overfitting and poor generalization to unseen architectures. In this work, we propose a fundamentally different formulation that models performance prediction as a conditional function inference problem using a Convolutional Neural Process (ConvNP) with meta-learning capabilities. Instead of fitting a fixed mapping to limited samples, our approach meta-learns to infer performance from partial observations by training with context-target splits across a group of synthesized tasks, explicitly optimizing for generalization under data scarcity and aligning the training procedure with the deployment setting in NAS. We further design simple yet effective meta-features for cell-based architectures and evaluate our method on NAS-Bench-101 and NAS-Bench-201. Extensive experiments show that our approach consistently improves top-K ranking quality and achieves the state-of-the-art architecture selection using limited samples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。