融合卷积与注意力机制，提升艺术品识别准确率
A Fusion Model for Artwork Identification Based on Convolutional Neural Networks and Transformers
- 用CNN提取局部特征，Transformer捕捉全局上下文，再融合两者优势
- 在中画和油画面板上准确率分别提升9.7%和7.1%，F1值提高0.06和0.05
- 适合需要高精度艺术品分类的文保、艺术市场研究场景
艺术品识别在文化遗产保护、艺术市场分析和历史研究中至关重要。随着深度学习发展，卷积神经网络（CNN）和Transformer模型已成为图像分类的关键工具。虽然CNN擅长局部特征提取，但难以捕捉全局上下文；而Transformer虽能建模全局依赖，却对细粒度局部细节表现较弱。为此，本文提出一种融合CNN与Transformer的艺术品识别模型：先通过CNN提取局部特征，再用Transformer捕获全局上下文，最后通过特征融合机制提升分类性能。在中文画作和油画数据集上的实验表明，该融合模型优于单独的CNN和Transformer模型，分类准确率分别提升9.7%和7.1%，F1分数分别提高0.06和0.05。结果验证了模型的有效性，未来可拓展至多模态融合与结构优化。
原文摘要 · Abstract (English)
The identification of artwork is crucial in areas like cultural heritage protection, art market analysis, and historical research. With the advancement of deep learning, Convolutional Neural Networks (CNNs) and Transformer models have become key tools for image classification. While CNNs excel in local feature extraction, they struggle with global context, and Transformers are strong in capturing global dependencies but weak in fine-grained local details. To address these challenges, this paper proposes a fusion model combining CNNs and Transformers for artwork identification. The model first extracts local features using CNNs, then captures global context with a Transformer, followed by a feature fusion mechanism to enhance classification accuracy. Experiments on Chinese and oil painting datasets show the fusion model outperforms individual CNN and Transformer models, improving classification accuracy by 9.7% and 7.1%, respectively, and increasing F1 scores by 0.06 and 0.05. The results demonstrate the model's effectiveness and potential for future improvements, such as multimodal integration and architecture optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。