用高效架构搜索找到2D/3D视觉平衡点，适合边缘设备部署
Dual-Domain Representation Alignment: Bridging 2D and 3D Vision via Geometry-Aware Architecture Search
- 基于混合超网络与双域知识蒸馏，提升模型表征能力
- 在多个数据集上实现准确率与效率的帕累托最优
- 支持多卡并发评估，推理延迟降低70%以上
现代计算机视觉需在预测精度与实时效率间取得平衡，但大型视觉模型（LVMs）的高推理成本限制了其在资源受限边缘设备上的部署。尽管进化神经架构搜索（ENAS）适用于多目标优化，但其实际应用受制于候选模型评估成本高和子网络排名不一致问题。为此，我们提出EvoNAS，一种高效的分布式多目标进化架构搜索框架。构建融合视觉状态空间与视觉变压器（VSS-ViT）模块的混合超网络，并采用跨架构双域知识蒸馏（CA-DDKD）策略进行优化。通过结合VSS块的计算效率与ViT模块的语义表达力，CA-DDKD提升了共享超网络的表征能力并增强排名一致性，实现无需额外微调的可靠适应度评估。为降低大规模验证开销，进一步提出基于GPU资源池化与异步调度的分布式多模型并行评估（DMMPE）框架。相比传统数据并行评估，DMMPE通过多卡多模型并发执行，效率提升超过70%。在COCO、ADE20K、KITTI和NYU-Depth v2数据集上的实验表明，所搜索出的EvoNets始终在准确率与效率间达成帕累托最优。相较于代表性CNN、ViT和Mamba基线模型，EvoNets在严格计算预算下具备更低推理延迟与更高吞吐量，且在新视角合成等下游任务中保持强泛化能力。代码已开源。
原文摘要 · Abstract (English)
Modern computer vision requires balancing predictive accuracy with real-time efficiency, yet the high inference cost of large vision models (LVMs) limits deployment on resource-constrained edge devices. Although Evolutionary Neural Architecture Search (ENAS) is well suited for multi-objective optimization, its practical use is hindered by two issues: expensive candidate evaluation and ranking inconsistency among subnetworks. To address them, we propose EvoNAS, an efficient distributed framework for multi-objective evolutionary architecture search. We build a hybrid supernet that integrates Vision State Space and Vision Transformer (VSS-ViT) modules, and optimize it with a Cross-Architecture Dual-Domain Knowledge Distillation (CA-DDKD) strategy. By coupling the computational efficiency of VSS blocks with the semantic expressiveness of ViT modules, CA-DDKD improves the representational capacity of the shared supernet and enhances ranking consistency, enabling reliable fitness estimation during evolution without extra fine-tuning. To reduce the cost of large-scale validation, we further introduce a Distributed Multi-Model Parallel Evaluation (DMMPE) framework based on GPU resource pooling and asynchronous scheduling. Compared with conventional data-parallel evaluation, DMMPE improves efficiency by over 70% through concurrent multi-GPU, multi-model execution. Experiments on COCO, ADE20K, KITTI, and NYU-Depth v2 show that the searched architectures, termed EvoNets, consistently achieve Pareto-optimal trade-offs between accuracy and efficiency. Compared with representative CNN-, ViT-, and Mamba-based models, EvoNets deliver lower inference latency and higher throughput under strict computational budgets while maintaining strong generalization on downstream tasks such as novel view synthesis. Code is available at https://github.com/EMI-Group/evonas
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。