定位多模态大模型中的知识神经元，提升模型可解释性。
Identifying Multi-modal Knowledge Neurons in Pretrained Transformers via Two-stage Filtering
- 两阶段过滤：先用图像修复筛选激活差异，再用GradCAM优化定位
- 在MS COCO 2017上，生成结果在BLEU、ROUGE和BERTScore上均优于现有方法
- 适合研究模型可解释性与知识编辑的学者使用
近年来，大语言模型（LLMs）推动了多模态大模型（MLLMs）的发展，实现了视觉与语言的融合理解。然而，这些模型存在内部机制不透明、易产生幻觉等问题。为此，本文提出一种基于MiniGPT-4的两阶段方法，识别特定知识对应的神经元。第一阶段通过图像修复（inpainting）进行激活差异过滤，第二阶段利用梯度加权类激活图（GradCAM）进行精细化筛选。在MS COCO 2017图像描述生成任务上，结合BLEU、ROUGE和BERTScore的定量评估，以及激活热力图的定性分析，验证了该方法在知识定位精度上优于现有技术。本研究有助于揭示多模态模型内部知识分布，为未来知识编辑与控制提供支持。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have led to the development of multimodal LLMs (MLLMs) in the fields of natural language processing (NLP) and computer vision. Although these models allow for integrated visual and language understanding, they present challenges such as opaque internal processing and the generation of hallucinations and misinformation. Therefore, there is a need for a method to clarify the location of knowledge in MLLMs. In this study, we propose a method to identify neurons associated with specific knowledge using MiniGPT-4, a Transformer-based MLLM. Specifically, we extract knowledge neurons through two stages: activation differences filtering using inpainting and gradient-based filtering using GradCAM. Experiments on the image caption generation task using the MS COCO 2017 dataset, BLEU, ROUGE, and BERTScore quantitative evaluation, and qualitative evaluation using an activation heatmap showed that our method is able to locate knowledge with higher accuracy than existing methods. This study contributes to the visualization and explainability of knowledge in MLLMs and shows the potential for future knowledge editing and control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。