arXiv:2605.27144cs.CVcs.LG2026-05

用超像素块构建视觉变压器,让图像分类更高效准确。

Is an Image Also Worth 16x16=256 Superpixels? A Framework for Attentional Image Classification

论文配图:Is an Image Also Worth 16x16=256 Superpixels? A Framework for Attentional Image Classification
图 1 · 摘自论文原文
  • 将超像素与视觉变压器结合,支持任意分块和图结构
  • 在多个数据集上超越传统超像素GNN方法,接近ViT性能
  • 适合关注图像分割与注意力机制融合的研究者

基于超像素的图像分类传统上使用图神经网络(GNN)处理不规则图像表示。近年来,受视觉变换器(ViTs)推动,自注意力模型在多种任务中超越卷积神经网络(CNN)。然而,GNN、超像素与变换器之间的协同关系尚未探索。本文提出超像素变换器(SPT),一种统一超像素图像分类与ViTs的新框架。SPT推广了超像素图像分类图注意力网络(SICGAT)模型和ViT,支持任意超像素分块策略、连接图结构和位置编码。引入多维正弦-余弦位置编码及增强的补丁数据结构,完整保留超像素形状与颜色信息。在CIFAR10、FashionMNIST和Imagenette等数据集上测试,采用不同超像素生成与图连接策略,结果表明SPT性能优于以往超像素GNN方法,并保持与ViTs相当的竞争力。该方法解决了SICGAT在像素聚合中的信息损失问题,揭示受限图连通性可提升ViT表现。SPT弥合了超像素与变换器模型间的鸿沟,为跨领域泛化与混合注意力框架的未来创新开辟路径,验证了图像也可价值$16\times16=256$个超像素。

原文摘要 · Abstract (English)

Superpixel-based image classification has traditionally leveraged graph neural networks (GNNs) for processing irregular image representations. Recent advances in computer vision, driven by Vision Transformers (ViTs), have introduced new paradigms in self-attentional models, surpassing convolutional neural networks (CNNs) in various tasks. However, a synergistic connection between GNNs, superpixels, and transformers remains unexplored. In this work, we propose Superpixel Transformers (SPT), a novel framework that unifies superpixel-based image classification and ViTs. SPT generalizes the Superpixel Image Classification with Graph Attention Networks (SICGAT) model and ViT to support arbitrary superpixel-based chunking strategies, connectivity graphs, and positional encodings. We introduce refinements including a multidimensional sine-cosine positional encoding and an enriched patch data structure that fully incorporates superpixel shape and color information. By testing SPT across datasets such as CIFAR10, FashionMNIST, and Imagenette, with various superpixel generation and graph connectivity strategies, we demonstrate that SPT achieves superior performance compared to previous superpixel-based GNN methods and remains competitive with ViTs. Notably, our approach addresses the limitations of SICGAT, such as information loss during pixel aggregation, and shows how constrained graph connectivity can enhance ViT performance. SPT bridges the gap between superpixel-based and transformer models, opening avenues for cross-domain generalization and future innovations in hybrid attentional frameworks, and showing that an image can also be worth $16\times16$ superpixels.

超像素视觉变换器注意力机制图像分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。