MIDAS用输入自适应注意力动态调整网络结构，提升搜索效率与精度。
MIDAS: Mosaic Input-Specific Differentiable Architecture Search
- 基于自注意力生成输入相关的动态架构参数，替代传统静态参数。
- 在CIFAR-10上达97.42%准确率，于NAS-Bench-201中稳定找到全局最优结构。
- 无需额外参数，支持局部空间分块搜索，适合高精度神经网络自动设计。
可微神经架构搜索（DARTS）虽提供高效的梯度优化方法，但实际应用受限。本文提出MIDAS，通过自注意力机制将静态架构参数替换为动态、输入相关的参数，实现更灵活的结构选择。为增强鲁棒性，MIDAS采用两个关键改进：(i) 对激活图每个空间块分别计算架构选择，实现局部化；(ii) 引入无参、拓扑感知的搜索空间，建模节点连接关系，并简化每节点选择两条输入边的过程。我们在DARTS、NAS-Bench-201和RDARTS搜索空间上评估MIDAS：在DARTS中，于CIFAR-10达到97.42% top-1准确率，在CIFAR-100达83.38%；在NAS-Bench-201中持续发现全局最优架构；在RDARTS的四个搜索空间中，于其中两个实现当前最佳性能。进一步分析表明，分块注意力增强了候选操作的区分能力，且所得输入相关参数分布具有类别感知性，主要呈单峰特征，为解码提供可靠指导。
原文摘要 · Abstract (English)
Differentiable Neural Architecture Search (NAS) provides efficient, gradient-based methods for automatically designing neural networks, yet its adoption remains limited in practice. We present MIDAS, a novel approach that modernizes DARTS by replacing static architecture parameters with dynamic, input-specific parameters computed via self-attention. To improve robustness, MIDAS (i) localizes the architecture selection by computing it separately for each spatial patch of the activation map, and (ii) introduces a parameter-free, topology-aware search space that models node connectivity and simplifies selecting the two incoming edges per node. We evaluate MIDAS on the DARTS, NAS-Bench-201, and RDARTS search spaces. In DARTS, it reaches 97.42% top-1 on CIFAR-10 and 83.38% on CIFAR-100. In NAS-Bench-201, it consistently finds globally optimal architectures. In RDARTS, it sets the state of the art on two of four search spaces on CIFAR-10. We further analyze why MIDAS works, showing that patchwise attention improves discrimination among candidate operations, and the resulting input-specific parameter distributions are class-aware and predominantly unimodal, providing reliable guidance for decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。