arXiv:2605.14047cs.CVcs.AR2026-05

用遗传算法为ViT每层定制缩放函数,省去重训练还能保性能。

Evolving Layer-Specific Scalar Functions for Hardware-Aware Transformer Adaptation

论文配图:Evolving Layer-Specific Scalar Functions for Hardware-Aware Transformer Adaptation
图 1 · 摘自论文原文
  • 用遗传编程为各层演化不同缩放函数,适配硬件特性。
  • 在20轮微调后恢复ImageNet-1K 84.32%准确率,捕获90%方差。
  • 无需重训练,显著降低计算量和内存流量,适合边缘部署。

视觉变压器(ViTs)在复杂视觉任务中表现卓越,但其在边缘设备上的部署受制于层归一化带来的计算复杂性和全局归约瓶颈。现有方法尝试用硬件友好的标量近似替代归一化层,但这些同质化替换无法适配所有层的行为,且依赖昂贵的模型重训练。本文提出一种高效、硬件感知的框架,利用遗传编程(GP)从预训练权重直接演化异构的、层特定的标量函数。结合新颖的后训练重对齐策略,该方法完全避免了从头重训练。所演化的表达式精准逼近目标归一化行为,相比同质基线的70.2%方差解释率,达到90%的$R^2$值。修改后的架构仅用20个周期即恢复84.32%的Top-1 ImageNet-1K准确率。在保留性能的同时消除全局归约瓶颈,相较标准层归一化,实现了严格的算术复杂度和片外内存流量降低,扫除了ViTs在边缘加速器上高效部署的主要障碍。

原文摘要 · Abstract (English)

Vision Transformers (ViTs) achieve state-of-the-art performance on challenging vision tasks, but their deployment on edge devices is severely hindered by the computational complexity and global reduction bottleneck imposed by layer normalization. Recent methods attempt to bypass this by replacing normalization layers with hardware-friendly scalar approximations. However, these homogeneous replacements do not optimally fit to all layers' behaviour and rely on expensive model retraining. In this work, we propose a highly efficient, hardware-aware framework that utilizes genetic programming (GP) to evolve heterogeneous, layer-specific scalar functions directly from pre-trained weights. Coupled with a novel post-training re-alignment strategy, our approach eliminates the need to retrain models from scratch entirely. Our evolved expressions accurately approximate the target normalization behaviours, capturing $90\%$ of the variance ($R^2$) compared to only $70.2\%$ for homogeneous baselines, allowing our modified architecture to recover $84.32\%$ Top-1 ImageNet-1K accuracy in only 20 epochs. By preserving this performance while eliminating the global reduction bottleneck, our approach achieves a strict reduction in both arithmetic complexity and off-chip memory traffic compared to standard LayerNorm, removing a primary barrier to the efficient deployment of ViTs on edge accelerators.

ViT优化边缘计算遗传编程硬件感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。