融合图像与文本的多标签分类模型，提升复杂图像的标签预测准确率。
Google is all you need: Semi-Supervised Transfer Learning Strategy For Light Multimodal Multi-Task Classification Model
- 用CNN处理图像，NLP分析描述文本，通过融合模块协同识别
- 支持1到19个标签（不含12），在多标签场景下准确率显著提升
- 适用于需要自动标注复杂图像的场景，如社交媒体内容管理
随着数字图像数据量的增长，图像分类的有效性不断提升。本研究提出一个稳健的多标签分类系统，可为单张图像分配多个标签，解决图像可能涉及多个类别（范围从1到19，不包括12）的复杂性问题。我们设计了一个多模态分类器，将先进的图像识别算法与自然语言处理（NLP）模型结合，并引入融合模块整合不同模态信息。引入文本数据旨在通过提供视觉分析无法完全捕捉的上下文理解，提升标签预测精度。所提模型结合卷积神经网络（CNN）进行图像处理，以及NLP技术分析图像描述（即标题）。该方法包含严格的训练与验证阶段，各组件均通过消融实验进行验证与分析。初步结果表明，该分类器具备高准确率与高效性，展现出作为自动图像标注系统的巨大潜力。
原文摘要 · Abstract (English)
As the volume of digital image data increases, the effectiveness of image classification intensifies. This study introduces a robust multi-label classification system designed to assign multiple labels to a single image, addressing the complexity of images that may be associated with multiple categories (ranging from 1 to 19, excluding 12). We propose a multi-modal classifier that merges advanced image recognition algorithms with Natural Language Processing (NLP) models, incorporating a fusion module to integrate these distinct modalities. The purpose of integrating textual data is to enhance the accuracy of label prediction by providing contextual understanding that visual analysis alone cannot fully capture. Our proposed classification model combines Convolutional Neural Networks (CNN) for image processing with NLP techniques for analyzing textual description (i.e., captions). This approach includes rigorous training and validation phases, with each model component verified and analyzed through ablation experiments. Preliminary results demonstrate the classifier's accuracy and efficiency, highlighting its potential as an automatic image-labeling system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。