arXiv:2603.00854cs.LGcs.IR2026-03

基于图神经网络的多模态推荐系统,助力濒危叙事卷轴画的发现与保护

GeMi: A Graph-based, Multimodal Recommendation System for Narrative Scroll Paintings

  • 构建图神经网络,融合文本与图像多模态特征进行内容推荐
  • 在真实田野调研数据集上实现高精度相似画作推荐,支持艺术传承
  • 适用于濒危艺术保护、文化存档及对传统绘画感兴趣的用户

推荐系统在管理日益增长的多模态数据方面表现优异,帮助用户发现新内容。这类系统可处理图像、文本、音频和视频等多种媒体类型,通过提取项目特征并结合用户偏好实现基于内容的推荐。基于图神经网络(GNN)的推荐系统能有效建模用户与项目间的关系,特别适合内容推荐任务。它们利用深度学习技术处理图结构数据,支持节点获取高阶邻居信息,并结合先进的视觉-语言模型处理多模态内容,具备完善的嵌入、消息传递与传播算法。本文提出一种针对濒危表演艺术形式的新型数据集上的GNN推荐系统设计,使用文本与图像多模态数据为用户提供相似卷轴画的观看与购买建议。据我们所知,目前尚无专为叙事卷轴画设计的推荐系统。本工作兼具艺术保护、濒危艺术品数据存档及先进推荐功能,充分挖掘了该数据的独特特性和目标用户群体偏好。

原文摘要 · Abstract (English)

Recommendation Systems are effective in managing the ever-increasing amount of multimodal data available today and help users discover interesting new items. These systems can handle various media types such as images, text, audio, and video data, and this has made it possible to handle content-based recommendation utilizing features extracted from items while also incorporating user preferences. Graph Neural Network (GNN)-based recommendation systems are a special class of recommendation systems that can handle relationships between items and users, making them particularly attractive for content-based recommendations. Their popularity also stems from the fact that they use advanced machine learning techniques, such as deep learning on graph-structured data, to exploit user-to-item interactions. The nodes in the graph can access higher-order neighbor information along with state-of-the-art vision-language models for processing multimodal content, and there are well-designed algorithms for embedding, message passing, and propagation. In this work, we present the design of a GNN-based recommendation system on a novel data set collected from field research. Designed for an endangered performing art form, the recommendation system uses multimodal content (text and image data) to suggest similar paintings for viewing and purchase. To the best of our knowledge, there is no recommendation system designed for narrative scroll paintings -- our work therefore serves several purposes, including art conservation, a data storage system for endangered art objects, and a state-of-the-art recommendation system that leverages both the novel characteristics of the data and preferences of the user population interested in narrative scroll paintings.

多模态推荐图神经网络艺术保护叙事卷轴画

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。