跨任务搜索新方法，提升模型迁移效率
Multi-Task Neural Architecture Search Using Architecture Embedding and Transfer Rank
- 用架构嵌入向量表示网络结构，实现无数据依赖的跨任务搜索
- 在多个数据集上优于同类方法，搜索效率与下游性能双提升
- 引入迁移秩机制，解决源任务与目标任务排名错位问题
多任务神经架构搜索（NAS）可实现不同任务间的架构知识迁移。然而，源任务与目标任务之间的排名不一致会降低架构在下游任务上的性能。本文提出KTNAS，一种基于进化策略的跨任务NAS算法，以提升迁移效率。该方法无需依赖数据，将神经网络架构转换为图结构，并利用架构嵌入向量进行性能预测。引入基于实例的分类器“迁移秩”机制，有效缓解性能下降问题。在NASBench-201上验证了搜索效率，在Micro TransNAS-Bench-101上展示了对多种视觉任务的迁移能力。在包含CIFAR-10/100、MNIST/Fashion-MNIST、MedMNIST的DARTs搜索空间中验证了方法可扩展性。实验结果表明，KTNAS在搜索效率和下游任务性能上均优于现有同类方法。消融实验证明迁移秩对迁移性能至关重要。
原文摘要 · Abstract (English)
Multi-task neural architecture search (NAS) enables transferring architectural knowledge among different tasks. However, ranking disorder between the source task and the target task degrades the architecture performance on the downstream task. We propose KTNAS, an evolutionary cross-task NAS algorithm, to enhance transfer efficiency. Our data-agnostic method converts neural architectures into graphs and uses architecture embedding vectors for the subsequent architecture performance prediction. The concept of transfer rank, an instance-based classifier, is introduced into KTNAS to address the performance degradation issue. We verify the search efficiency on NASBench-201 and transferability to various vision tasks on Micro TransNAS-Bench-101. The scalability of our method is demonstrated on DARTs search space including CIFAR-10/100, MNIST/Fashion-MNIST, MedMNIST. Experimental results show that KTNAS outperforms peer multi-task NAS algorithms in search efficiency and downstream task performance. Ablation studies demonstrate the vital importance of transfer rank for transfer performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。