通过替换外部记忆库实现零样本分类，无需重训练模型。
Memory-Modular Classification: Learning to Generalize with Memory Replacement
- 将知识存入网络外的网页图文记忆库，推理时动态检索
- 更换记忆内容即可适应新类别，零样本分类准确率超80%
- 适合需要快速适应新类别的实际场景，如增量学习
我们提出一种新型的内存模块化图像分类学习器，将知识记忆与推理过程分离。该模型通过简单替换外部记忆内容即可实现对新类别的有效泛化，无需重新训练模型。不同于传统模型在训练过程中将世界知识和任务特定技能编码到权重中，本模型将知识存储于从网络爬取的图像和文本数据构成的外部记忆中。推理时，模型根据输入图像动态选择相关记忆内容，从而通过更换记忆内容即可适应任意类别。其核心优势在于：元学习如何利用未见类别的噪声网络数据完成分类任务，在多种分类场景下均表现出鲁棒性能。实验结果表明，该方法在处理多样化的分类任务方面具有显著优势，包括对未见类别的零样本/少样本分类、细粒度分类以及类别增量分类。
原文摘要 · Abstract (English)
We propose a novel memory-modular learner for image classification that separates knowledge memorization from reasoning. Our model enables effective generalization to new classes by simply replacing the memory contents, without the need for model retraining. Unlike traditional models that encode both world knowledge and task-specific skills into their weights during training, our model stores knowledge in the external memory of web-crawled image and text data. At inference time, the model dynamically selects relevant content from the memory based on the input image, allowing it to adapt to arbitrary classes by simply replacing the memory contents. The key differentiator that our learner meta-learns to perform classification tasks with noisy web data from unseen classes, resulting in robust performance across various classification scenarios. Experimental results demonstrate the promising performance and versatility of our approach in handling diverse classification tasks, including zero-shot/few-shot classification of unseen classes, fine-grained classification, and class-incremental classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。