用知识图谱追踪生成图像的训练数据来源，提升模型透明度。
Training Data Attribution for Image Generation using Ontology-Aligned Knowledge Graphs
- 通过多模态大模型从图像提取结构化三元组，构建对齐领域本体的知识图谱。
- 对比生成图与训练图的知识图谱，可识别潜在数据影响，支持版权分析。
- 适用于需解释生成内容来源的研究者与开发者，推动可信AI发展。
随着生成模型能力增强，透明度、责任归属和版权问题日益突出。理解特定训练数据如何影响模型输出至关重要。本文提出一种通过自动构建对齐本体的知识图谱（KG）来解释生成结果的框架。尽管自然文本的自动知识图谱构建已取得进展，但从视觉内容中提取结构化且符合本体一致性的表示仍具挑战，原因在于图像信息丰富且包含多个对象。借助多模态大语言模型（LLMs），我们的方法从图像中提取与领域本体对齐的结构化三元组。通过比较生成图像与训练图像的知识图谱，可追溯潜在影响，实现版权分析、数据集透明化与可解释性。我们在本地训练模型的去训练实验以及大规模模型的风格特异性实验中验证了该方法的有效性。该框架有助于构建促进人类协作、激发创造力与好奇心的AI系统。
原文摘要 · Abstract (English)
As generative models become powerful, concerns around transparency, accountability, and copyright violations have intensified. Understanding how specific training data contributes to a model's output is critical. We introduce a framework for interpreting generative outputs through the automatic construction of ontologyaligned knowledge graphs (KGs). While automatic KG construction from natural text has advanced, extracting structured and ontology-consistent representations from visual content remains challenging -- due to the richness and multi-object nature of images. Leveraging multimodal large language models (LLMs), our method extracts structured triples from images, aligned with a domain-specific ontology. By comparing the KGs of generated and training images, we can trace potential influences, enabling copyright analysis, dataset transparency, and interpretable AI. We validate our method through experiments on locally trained models via unlearning, and on large-scale models through a style-specific experiment. Our framework supports the development of AI systems that foster human collaboration, creativity and stimulate curiosity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。