arXiv:2511.18691cs.CV2025-11

融合三类视觉模型,提升分类精度并降低计算成本

EVCC: Enhanced Vision Transformer-ConvNeXt-CoAtNet Fusion for Classification

  • 采用多分支结构融合Transformer、ConvNeXt与CoAtNet
  • 在多个数据集上准确率领先,计算量减少25%~35%
  • 动态调整计算资源,适合实际部署场景

混合视觉架构结合Transformer与CNN在图像分类上取得显著进展,但通常伴随高昂的计算开销。本文提出EVCC(增强型视觉Transformer-ConvNeXt-CoAtNet融合模型),通过四项创新实现高效融合:(1) 自适应标记剪枝以保留信息,(2) 门控双向交叉注意力增强特征优化,(3) 辅助分类头支持多任务学习,(4) 基于上下文感知置信度加权的动态路由门。在CIFAR-100、Tobacco3482、CelebA和Brain Cancer数据集上的实验表明,EVCC在保持甚至超越DeiT-Base、MaxViT-Base、CrossViT-Base等强大模型性能的同时,准确率最高提升2个百分点,且计算量(FLOPs)降低25%至35%。其自适应架构可根据部署需求动态减少标记数量,有效平衡精度与效率,同时融合全局上下文、局部细节与层级特征,适用于真实应用场景。代码已公开于https://anonymous.4open.science/r/EVCC。

原文摘要 · Abstract (English)

Hybrid vision architectures combining Transformers and CNNs have significantly advanced image classification, but they usually do so at significant computational cost. We introduce EVCC (Enhanced Vision Transformer-ConvNeXt-CoAtNet), a novel multi-branch architecture integrating the Vision Transformer, lightweight ConvNeXt, and CoAtNet through key innovations: (1) adaptive token pruning with information preservation, (2) gated bidirectional cross-attention for enhanced feature refinement, (3) auxiliary classification heads for multi-task learning, and (4) a dynamic router gate employing context-aware confidence-driven weighting. Experiments across the CIFAR-100, Tobacco3482, CelebA, and Brain Cancer datasets demonstrate EVCC's superiority over powerful models like DeiT-Base, MaxViT-Base, and CrossViT-Base by consistently achieving state-of-the-art accuracy with improvements of up to 2 percentage points, while reducing FLOPs by 25 to 35%. Our adaptive architecture adjusts computational demands to deployment needs by dynamically reducing token count, efficiently balancing the accuracy-efficiency trade-off while combining global context, local details, and hierarchical features for real-world applications. The source code of our implementation is available at https://anonymous.4open.science/r/EVCC.

图像分类多模型融合轻量化设计动态计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。