用四向分层存贮机制实现高效图像分类,参数少、速度快。
HiRo: A Compact Four-Directional Hierarchical Reservoir Token-Mixer for Efficient Image Classification

- 采用四方向扫描与分片混合的固定存贮模块,减少可训练参数。
- 在MNIST/CIFAR-10/CIFAR-100上分别达99.46%/85.57%/59.10%准确率。
- 适合追求低参数、低延迟的轻量级图像分类任务。
当前图像分类模型需平衡局部特征建模、跨窗交互与参数效率。许多高性能架构依赖全可训练的令牌混合器,虽提升表征能力但增加参数量、优化复杂度和计算开销。本文提出一种参数高效的图像分类模型HiRo,结合移位窗口划分与多方向分层存贮计算。图像被划分为非重叠块(视为令牌),线性投影、归一化并添加二维正弦位置编码后,在局部窗口内处理。每个窗口内,令牌按四个方向扫描,经两阶段切片-混合存贮模块处理:第一阶段将方向序列切分为连续片段,各片段由独立固定存贮单元处理,配合可训练闭环读出;输出通过起始、结尾、均值表示汇总,并由第二阶段固定存贮单元进行方向内混合;混合后的片段表示还原至令牌层级,与第一阶段输出融合,四方向结果对齐平均。相邻块交替使用常规与移位窗口以实现跨窗交互,后续经层归一化、残差前馈网络及全局池化完成分类。该设计融合常规与移位窗口划分,构建高效局部到跨窗的令牌混合框架。尽管仅使用不到100万可训练参数,内存与时间消耗显著低于基于Transformer的基线,仍达到在MNIST、CIFAR-10、CIFAR-100上的99.46%、85.57%、59.10%准确率。
原文摘要 · Abstract (English)
Recent image classification models must balance local feature modeling, cross-window interaction, and parameter efficiency. Many high-performing architectures rely on fully trainable token-mixers, which improve representation learning but increase parameter count, optimization complexity and computational cost. We propose a parameter-efficient image classification model called HiRo that integrates shifted-window partitioning with multi-directional hierarchical reservoir computing. Images are divided into non-overlapping patches (treated as tokens), linearly projected, normalized, and enriched with 2D sinusoidal positional encodings, then processed within local windows. Inside each window, tokens are scanned in four directions and passed through a two-stage slice-and-mix reservoir module. In the first stage, directional sequences are split into contiguous slices, each processed by its own fixed reservoir with a trainable closed-loop readout. The resulting slice outputs are summarized using the start, end, and mean representations, and then mixed by a second-stage fixed reservoir for each direction. The mixed slice representations are expanded back to the token level and fused with the first-stage outputs, after which the four directional outputs are realigned and averaged. Consecutive blocks alternate between regular and shifted windows to enable cross-window interaction, followed by layer normalization, a residual feed-forward network, and global pooling for classification. This design combines regular and shifted window partitioning with hierarchical multi-directional reservoirs to make an efficient local-to-cross-window token-mixing framework for image classification. Despite using under 1M trainable parameters and significantly lower memory and time than transformer-style baselines, HiRo also achieves 99.46%, 85.57%, and 59.10% accuracy on MNIST, CIFAR-10, and CIFAR-100, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。