arXiv:2605.01742cs.CV2026-05

联合优化视觉变压器的结构、令牌和位宽,提升芯片封装检测效率

Joint Architecture-Token-Bitwidth Multi-Axis Optimization of Vision Transformers for Semiconductor IC Packaging

论文配图:Joint Architecture-Token-Bitwidth Multi-Axis Optimization of Vision Transformers for Semiconductor IC Packaging
图 1 · 摘自论文原文
  • 三轴协同优化:结构搜索+令牌压缩+混合精度推理
  • 吞吐量提升超10倍,参数、计算量和能耗均减少10倍以上
  • 专为半导体制造场景设计,兼顾高精度与低资源消耗

视觉变压器(ViTs)在图像识别中表现优异,但其在资源受限的工业环境中部署仍受制约,主要问题在于计算开销大、内存占用高、能耗高。尽管神经架构搜索(NAS)、令牌压缩和低精度推理等单轴优化技术已被广泛研究,但多数工作仅聚焦单一维度,难以实现整体性能提升。本文提出首个联合优化架构、令牌和位宽三个维度的综合框架。通过AutoFormer生成紧凑骨干网络,采用ToMe方法合并令牌以减少信息处理量,并使用fp16混合精度推理加速运算。基于DeiT-B/16基准,在ImageNet-1K上分析极端压缩下的精度-效率权衡后,将所选配置应用于自研3D X-ray半导体缺陷分类数据集进行芯片封装检测。结果表明,该多轴框架在保持下游工业任务所需精度的前提下,实现超过10倍的吞吐量提升,参数量、浮点运算量(FLOPs)及能耗均降低超过10倍。据我们所知,这是最早联合优化ViT三维度的成果之一,也是首个面向半导体制造场景的资源高效部署研究。

原文摘要 · Abstract (English)

Vision Transformers (ViTs) have achieved strong performance in visual recognition, yet their deployment in resource-constrained industrial environments remains limited. Some main challenges are their high computational cost, memory requirement, and energy consumption. While individual efficiency techniques such as neural architecture search (NAS), token compression, and low-precision inference have been extensively studied, most prior work targets only a single optimization axis, limiting overall deployment gains while preserving accuracy. In this paper, we present one of the first holistic frameworks that jointly optimizes three complementary axes: architecture, token, and bit-width. Specifically, the framework identifies compact backbones via Neural Architecture Search (AutoFormer), reduces information processing via token merging (ToMe), and accelerates per-operation execution via fp16 mixed-precision inference. Starting from a DeiT-B/16 baseline, we first analyze accuracy-efficiency trade-offs on ImageNet-1K under aggressive compression. Then, we apply the selected configurations to a real-world in-house 3D X-ray semiconductor defect classification dataset for IC chip packaging inspection. Results show that the proposed multi-axis framework achieves more than 10 times improvement in throughput along with over 10 times reductions in parameter count, FLOPs, and energy consumption, while maintaining the required accuracy on the downstream industrial task. To the best of our knowledge, this is among the earliest works to jointly optimize architecture, token, and bit-width dimensions in ViTs and the first such resource-efficient, deployment-focused study tailored to semiconductor manufacturing.

视觉变压器芯片检测多轴优化轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。