通过语义感知重分块，让图像识别更快更准。
ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition
- 按语义和局部一致性重组图像块，提升语义表达
- DeiT-S推理速度提升54%，251%加速下精度超基线8%以上
- 兼容现有压缩方法，适合高效视觉模型部署
视觉变压器（ViTs）在各类计算机视觉任务中表现优异，但其依赖多头自注意力机制导致计算开销巨大，限制了实际应用。现有工作主要通过减少令牌数量来加速，但由于原始令牌来自非重叠网格块，语义信息不足，难以适配高效ViT。为此，我们提出ImagePiece，一种面向视觉变压器的新型重分块策略。受自然语言处理中MaxMatch分词思想启发，ImagePiece将语义不充分但局部一致的令牌合并，直到其具备语义意义。该简单重分块方法高度兼容已有令牌压缩技术，能大幅缩小相关令牌范围，在DeiT-S上实现54%的推理加速（近1.5倍），同时图像分类准确率提升0.39%；在超高速推理场景（加速251%）下，精度超越其他基线超过8%。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) have achieved remarkable success in various computer vision tasks. However, ViTs have a huge computational cost due to their inherent reliance on multi-head self-attention (MHSA), prompting efforts to accelerate ViTs for practical applications. To this end, recent works aim to reduce the number of tokens, mainly focusing on how to effectively prune or merge them. Nevertheless, since ViT tokens are generated from non-overlapping grid patches, they usually do not convey sufficient semantics, making it incompatible with efficient ViTs. To address this, we propose ImagePiece, a novel re-tokenization strategy for Vision Transformers. Following the MaxMatch strategy of NLP tokenization, ImagePiece groups semantically insufficient yet locally coherent tokens until they convey meaning. This simple retokenization is highly compatible with previous token reduction methods, being able to drastically narrow down relevant tokens, enhancing the inference speed of DeiT-S by 54% (nearly 1.5$\times$ faster) while achieving a 0.39% improvement in ImageNet classification accuracy. For hyper-speed inference scenarios (with 251% acceleration), our approach surpasses other baselines by an accuracy over 8%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。