arXiv:2412.18178cs.CV2024-12

VisionGRU用线性复杂度RNN实现高效图像分类,适合高分辨率图像处理。

VisionGRU: A Linear-Complexity RNN Model for Efficient Image Analysis

  • 用简化GRU分块处理图像,逐步减少序列长度并增加通道深度。
  • 在ImageNet和ADE20K上优于ViT,内存和计算成本显著降低。
  • 适合需要高效推理的视觉任务,如语义分割与高分辨率图像分析。

卷积神经网络(CNN)和视觉变压器(ViTs)是图像分析的主流模型。尽管CNN擅长提取多尺度特征,而ViT能有效捕捉全局依赖关系,但两者在处理高分辨率图像时均存在计算开销大的问题。近年来,状态空间模型(SSMs)和循环神经网络(RNN)因效率高受到关注,但其在图像分类任务中的表现仍有限。为此,本文提出VisionGRU,一种基于RNN的新型架构,专为高效图像分类设计。VisionGRU采用简化的门控循环单元(minGRU),以线性复杂度处理大规模图像特征。它将图像划分为小块,逐步减少序列长度同时增加通道深度,从而实现多尺度特征提取。通过带有双向扫描的分层2DGRU模块,可同时捕获局部与全局上下文,提升长程依赖建模能力,尤其适用于语义分割等任务。在ImageNet和ADE20K数据集上的实验表明,VisionGRU性能优于ViT,显著降低内存占用与计算成本,尤其在高分辨率图像处理中优势明显。这些结果凸显了基于RNN方法在构建高效可扩展视觉系统方面的潜力。代码将公开于https://github.com/YangLiu9208/VisionGRU。

原文摘要 · Abstract (English)

Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) are two dominant models for image analysis. While CNNs excel at extracting multi-scale features and ViTs effectively capture global dependencies, both suffer from high computational costs, particularly when processing high-resolution images. Recently, state-space models (SSMs) and recurrent neural networks (RNNs) have attracted attention due to their efficiency. However, their performance in image classification tasks remains limited. To address these challenges, this paper introduces VisionGRU, a novel RNN-based architecture designed for efficient image classification. VisionGRU leverages a simplified Gated Recurrent Unit (minGRU) to process large-scale image features with linear complexity. It divides images into smaller patches and progressively reduces the sequence length while increasing the channel depth, thus facilitating multi-scale feature extraction. A hierarchical 2DGRU module with bidirectional scanning captures both local and global contexts, improving long-range dependency modeling, particularly for tasks like semantic segmentation. Experimental results on the ImageNet and ADE20K datasets demonstrate that VisionGRU outperforms ViTs, significantly reducing memory usage and computational costs, especially for high-resolution images. These findings underscore the potential of RNN-based approaches for developing efficient and scalable computer vision solutions. Codes will be available at https://github.com/YangLiu9208/VisionGRU.

图像分类RNN高效模型视觉建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。