用元学习加速联邦神经架构搜索，无需重训练即可高效找到适配异构数据的模型。
Federated Neural Architecture Search with Model-Agnostic Meta Learning
- 结合元学习与可微分搜索，动态调整模型权重和结构参数。
- 相比FedNAS提速超50%，且在异构数据下准确率更高。
- 支持快速剪枝，剪枝后模型可直接部署，省去重训练环节。
联邦学习因设备间数据分布不均而面临挑战。联邦神经架构搜索（Federated NAS）旨在协作寻找适配异构数据的最优模型结构以提升准确率，但搜索过程耗时且需频繁重训练。为此，本文提出FedMetaNAS框架，将元学习引入联邦环境中的神经架构搜索，通过剪枝搜索空间并移除重训练阶段来加速搜索。首先采用Gumbel-Softmax重参数化实现搜索空间中混合操作的连续松弛；接着引入模型无关元学习（Model-Agnostic Meta-Learning），使每个任务的局部学习器同时优化权重和结构参数（alphas），而元学习器则根据任务学习器的梯度信息更新全局权重与alphas。完成元更新后，利用相同技巧进行软剪枝，逐步稀疏化网络结构，确保剪枝后模型性能稳定，可直接使用而无需重训练。实验表明，与FedNAS相比，FedMetaNAS在异构数据场景下搜索速度提升超过50%，且准确率更高。
原文摘要 · Abstract (English)
Federated Learning (FL) often struggles with data heterogeneity due to the naturally uneven distribution of user data across devices. Federated Neural Architecture Search (NAS) enables collaborative search for optimal model architectures tailored to heterogeneous data to achieve higher accuracy. However, this process is time-consuming due to extensive search space and retraining. To overcome this, we introduce FedMetaNAS, a framework that integrates meta-learning with NAS within the FL context to expedite the architecture search by pruning the search space and eliminating the retraining stage. Our approach first utilizes the Gumbel-Softmax reparameterization to facilitate relaxation of the mixed operations in the search space. We then refine the local search process by incorporating Model-Agnostic Meta-Learning, where a task-specific learner adapts both weights and architecture parameters (alphas) for individual tasks, while a meta learner adjusts the overall model weights and alphas based on the gradient information from task learners. Following the meta-update, we propose soft pruning using the same trick on search space to gradually sparsify the architecture, ensuring that the performance of the chosen architecture remains robust after pruning which allows for immediate use of the model without retraining. Experimental evaluations demonstrate that FedMetaNAS significantly accelerates the search process by more than 50\% with higher accuracy compared to FedNAS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。