用轻量框架在消费级显卡上高效搜出小模型,兼顾精度与速度。
Transformer-Guided Swarm Intelligence for Frugal Neural Architecture Search

- 融合Transformer全局搜索与蜂群算法局部优化,提升搜索效率。
- 3小时内用17.4万参数达84.85%准确率,远小于主流模型。
- 适配金融反欺诈场景,小模型实现0.71 F1分数,适合边缘部署。
神经架构搜索(NAS)已能自动化设计深度学习模型,但传统方法需数千GPU天的算力。本文提出一种轻量且具有遗传特性的NAS框架,旨在让普通硬件也能进行架构设计。该方法结合自回归Transformer控制器(通过强化学习训练)的全局宏观搜索能力,以及人工蜂群算法(ABC)的局部微观优化能力。为避免强化学习阶段过早收敛,引入动态熵机制,在检测到性能停滞时强制拓扑探索。在标准显卡(NVIDIA RTX 3060)上评估,该混合方法有效解决元启发式算法的“冷启动”问题。通过算法惩罚网络深度,主动抑制模型膨胀:在CIFAR-10数据集上,仅用约17.4万参数即达到84.85%准确率(显著低于ResNet-20等标准基线),搜索时间仅3小时。此外,我们证明了该框架的灵活性,将其应用于信用卡欺诈检测任务,直接优化高度不平衡表格数据上的F1-Score,最终以约4,600参数的小模型达成0.71 F1分数。结果表明,该方法可生成适用于边缘部署的定制化、高参数效率深度模型。
原文摘要 · Abstract (English)
Neural Architecture Search (NAS) has automated the design of deep learning models but traditionally requires massive computational resources, often measured in thousands of GPU-days. In this paper, we propose a frugal and memetic NAS framework designed to democratize architecture design on consumer-grade hardware. Our approach combines the global macro-search capabilities of an autoregressive Transformer controller, trained via Reinforcement Learning (RL), with the local micro-exploitation of an Artificial Bee Colony (ABC) algorithm. To prevent premature convergence during the RL phase, we introduce a dynamic entropy mechanism that forces topological exploration upon detection of performance stagnation. Evaluated on a standard GPU (NVIDIA RTX 3060), our hybrid method effectively resolves the "cold-start" problem inherent in metaheuristics. By algorithmically penalizing network depth, our framework actively mitigates model bloat: on the CIFAR-10 dataset, it discovers an efficient architecture reaching 84.85% accuracy with only $\sim$174,000 parameters (significantly smaller than standard baselines like ResNet-20) in 3 hours of search time. Furthermore, we demonstrate the framework's flexibility by applying it to credit card fraud detection, directly optimizing the F1-Score on highly imbalanced tabular data to reach a F1-Score of 0.71 with a compact network of $\sim$4,600 parameters. These results suggest that our approach can yield tailored, accessible, and highly parameter-efficient deep learning models suitable for edge deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。