arXiv:2603.02047cs.CV2026-03被引 1

用多模态超图增强检索生成,帮公共卫生研究者分析成瘾产品

NICO-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Understanding the Nicotine Public Health Crisis

  • 构建图像与文本联合的超图结构,实现跨模态知识关联
  • 在不处理图像令牌情况下,100+问题回答效果媲美顶尖图像RAG方法
  • 适用于烟草产品研究、政策制定者及公共健康数据分析人员

尼古丁成瘾公共卫生危机持续蔓延。本世纪以来,烟草行业推出大量新型产品,如含味尼古丁袋,以吸引新用户。这些创新已抵消多年控烟努力成果。现有研究在数据规模和跨数据点关联能力上均有限。为此,我们构建了包含55个品牌、超过20万张图像与文本描述的多模态数据集NICO-Dataset。同时提出NICO-RAG框架,通过构建图像与文本抽取实体及关系的超图,在不需处理图像令牌的前提下实现高效检索。该联合多模态知识表示使系统不仅能基于视觉相似性,还能根据图像描述语义进行检索。实验表明,在100多个问题上,无需额外图像令牌处理,其性能可媲美当前最优图像适配型RAG方法。

原文摘要 · Abstract (English)

The nicotine addiction public health crisis continues to be pervasive. In this century alone, the tobacco industry has released and marketed new products in an aggressive effort to lure new and young customers for life. Such innovations and product development, namely flavored nicotine or tobacco such as nicotine pouches, have undone years of anti-tobacco campaign work. Past work is limited both in scope and in its ability to connect large-scale data points. Thus, we introduce the Nicotine Innovation Counter-Offensive (NICO) Dataset to provide public health researchers with over 200,000 multimodal samples, including images and text descriptions, on 55 tobacco and nicotine product brands. In addition, to provide public health researchers with factual connections across a large-scale dataset, we propose NICO-RAG, a retrieval-augmented generation (RAG) framework that can retrieve image features without incurring the high-cost of language models, as well as the added cost of processing image tokens with large-scale datasets such as NICO. At construction time, NICO-RAG organizes image- and text-extracted entities and relations into hypergraphs to produce as factual responses as possible. This joint multimodal knowledge representation enables NICO-RAG to retrieve images for query answering not only by visual similarity but also by the semantic similarity of image descriptions. Experimentals show that without needing to process additional tokens from images for over 100 questions, NICO-RAG performs comparably to the state-of-the-art RAG method adapted for images.

多模态超图信息检索公共卫生

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。