arXiv:2512.20112cs.NEcs.AI2025-12被引 1

用双对比学习提升神经架构搜索效率,仅需少量训练即可获得高精度预测器。

Evolutionary Neural Architecture Search with Dual Contrastive Learning

  • 分两阶段对比学习:先无监督学架构表征,再细调预测相对性能
  • 在NASBench-101上比最强基线高0.39%,图像分类任务上超手动设计模型2.5个百分点
  • 适合资源受限场景,7.7 GPU天即完成真实医疗数据任务优化

进化式神经架构搜索(ENAS)因其可自动设计神经网络架构而受到关注。现有研究使用神经预测器引导搜索过程,但获取训练数据的计算成本极高——每个标签都需要完整训练一个架构,因此在有限算力预算(即限定完全训练的架构-标签对数量)下,构建高精度预测器对ENAS成功至关重要。本文提出基于双对比学习的ENAS(DCL-ENAS),通过两个阶段的对比学习训练神经预测器:第一阶段采用对比自监督学习,在无需标签的情况下从神经架构中学习有意义的表征;第二阶段通过对比学习微调,准确预测不同架构的相对性能而非绝对性能,这已足以指导进化搜索。在NASBench-101和NASBench-201上,DCL-ENAS达到最高验证精度,超越最强公开基线0.05%(ImageNet16-120)至0.39%(NASBench-101)。在真实世界的心电图心律失常分类任务中,相较于通过随机搜索获得的手动设计非NAS模型,DCL-ENAS性能提升约2.5个百分点,且仅需7.7 GPU天。

原文摘要 · Abstract (English)

Evolutionary Neural Architecture Search (ENAS) has gained attention for automatically designing neural network architectures. Recent studies use a neural predictor to guide the process, but the high computational costs of gathering training data -- since each label requires fully training an architecture -- make achieving a high-precision predictor with { limited compute budget (i.e., a capped number of fully trained architecture-label pairs)} crucial for ENAS success. This paper introduces ENAS with Dual Contrastive Learning (DCL-ENAS), a novel method that employs two stages of contrastive learning to train the neural predictor. In the first stage, contrastive self-supervised learning is used to learn meaningful representations from neural architectures without requiring labels. In the second stage, fine-tuning with contrastive learning is performed to accurately predict the relative performance of different architectures rather than their absolute performance, which is sufficient to guide the evolutionary search. Across NASBench-101 and NASBench-201, DCL-ENAS achieves the highest validation accuracy, surpassing the strongest published baselines by 0.05\% (ImageNet16-120) to 0.39\% (NASBench-101). On a real-world ECG arrhythmia classification task, DCL-ENAS improves performance by approximately 2.5 percentage points over a manually designed, non-NAS model obtained via random search, while requiring only 7.7 GPU-days.

神经架构搜索对比学习进化算法低资源

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。