用扩散模型给分类模型加'反思能力',不训练就能提效。
An Efficient Framework for Enhancing Discriminative Models via Diffusion Techniques
- 不训练地融合生成与判别模型,让网络学会重新思考
- ResNet-50在ImageNet上提升1.51%,ImageNet-A上达3.02%
- 适配各类主流模型,插件式使用,尤其适合追求泛化性能的场景
图像分类是计算机视觉的核心任务,传统依赖深度神经网络的判别模型。近期基于生成模型的分类方法虽支持零样本分类,但存在计算开销大、性能低于判别模型的问题。受人脑快速-慢速通路协同识别机制启发,本文提出扩散增强判别模型框架(DBMEF),无需训练即可融合判别与生成模型:利用判别模型初判,再通过扩散模型赋予网络重思能力。该框架以即插即用方式显著提升判别模型的分类准确率与泛化能力。我们在17种主流模型(包括ResNet等CNN和ViT等Transformer)上验证了有效性,结果表明:ResNet-50在ImageNet上提升1.51%,在ImageNet-A上提升3.02%。研究揭示了一种新的图像分类范式,对不同数据集和网络结构均表现稳定增益。代码已开源:https://github.com/ChunXiaostudy/DBMEF。
原文摘要 · Abstract (English)
Image classification serves as the cornerstone of computer vision, traditionally achieved through discriminative models based on deep neural networks. Recent advancements have introduced classification methods derived from generative models, which offer the advantage of zero-shot classification. However, these methods suffer from two main drawbacks: high computational overhead and inferior performance compared to discriminative models. Inspired by the coordinated cognitive processes of rapid-slow pathway interactions in the human brain during visual signal recognition, we propose the Diffusion-Based Discriminative Model Enhancement Framework (DBMEF). This framework seamlessly integrates discriminative and generative models in a training-free manner, leveraging discriminative models for initial predictions and endowing deep neural networks with rethinking capabilities via diffusion models. Consequently, DBMEF can effectively enhance the classification accuracy and generalization capability of discriminative models in a plug-and-play manner. We have conducted extensive experiments across 17 prevalent deep model architectures with different training methods, including both CNN-based models such as ResNet and Transformer-based models like ViT, to demonstrate the effectiveness of the proposed DBMEF. Specifically, the framework yields a 1.51\% performance improvement for ResNet-50 on the ImageNet dataset and 3.02\% on the ImageNet-A dataset. In conclusion, our research introduces a novel paradigm for image classification, demonstrating stable improvements across different datasets and neural networks. The code is available at https://github.com/ChunXiaostudy/DBMEF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。