综述轻量级视觉Transformer的在线生成方法，助力高效图像识别。
Image Recognition with Online Lightweight Vision Transformer: A Survey
- 提出三种轻量化策略：高效组件设计、动态网络和知识蒸馏。
- 在ImageNet-1K上评估，精度与参数、吞吐量间实现可调平衡。
- 适合关注模型效率与部署落地的研究者和工程师。
Transformer架构在自然语言处理中取得显著成功，推动其向计算机视觉任务迁移。与卷积神经网络不同，视觉Transformer天然捕捉长程依赖并支持并行计算，但缺乏归纳偏置和效率优势，面临严重的计算与内存挑战，限制了实际应用。本文综述了用于图像识别的轻量级视觉Transformer的在线生成策略，聚焦三大方向：高效组件设计、动态网络结构与知识蒸馏。我们在ImageNet-1K基准上评估相关工作，分析精度、参数量、吞吐量等指标间的权衡，揭示各类方法的优势、局限与灵活性。最后，提出未来研究方向与潜在挑战，旨在激发进一步探索，并为社区提供实践指导。项目页面：https://github.com/ajxklo/Lightweight-VIT
原文摘要 · Abstract (English)
The Transformer architecture has achieved significant success in natural language processing, motivating its adaptation to computer vision tasks. Unlike convolutional neural networks, vision transformers inherently capture long-range dependencies and enable parallel processing, yet lack inductive biases and efficiency benefits, facing significant computational and memory challenges that limit its real-world applicability. This paper surveys various online strategies for generating lightweight vision transformers for image recognition, focusing on three key areas: Efficient Component Design, Dynamic Network, and Knowledge Distillation. We evaluate the relevant exploration for each topic on the ImageNet-1K benchmark, analyzing trade-offs among precision, parameters, throughput, and more to highlight their respective advantages, disadvantages, and flexibility. Finally, we propose future research directions and potential challenges in the lightweighting of vision transformers with the aim of inspiring further exploration and providing practical guidance for the community. Project Page: https://github.com/ajxklo/Lightweight-VIT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。