发现多模态大模型会无意记住无关隐私信息，哪怕它不影响回答结果。
Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Models
- 通过在图像中随机添加水印，测试模型是否无意识记忆无关内容。
- 部分批次训练下，模型对无关水印产生明显不同的学习行为。
- 即使不影响输出，模型仍会激活特定表征模式，暴露隐私记忆。
多模态大语言模型（MLLMs）在视觉问答（VQA）等任务上表现优异。尽管已有研究指出其可能记忆与任务相关的内容，但尚不清楚它们是否会无意中记住完全无关的私密内容。本文探究了随机生成的、与任务无关的私密内容如何因部分小批量训练动态而与下游目标产生虚假关联，导致无意记忆。具体而言，我们在VQA微调图像中以不同概率随机嵌入任务无关水印，并提出一种新型探测框架，判断MLLM是否无意编码此类内容。实验表明，当嵌入任务无关水印时，MLLM在部分小批量设置下表现出显著不同的训练行为。通过逐层探测，我们进一步证明，即使这些无关知识不改变模型输出，当再次遇到时，模型仍会触发独特的表征模式。代码已开源：https://github.com/illusionhi/ProbingPrivacy。
原文摘要 · Abstract (English)
Multi-Modal Large Language Models (MLLMs) have exhibited remarkable performance on various vision-language tasks such as Visual Question Answering (VQA). Despite accumulating evidence of privacy concerns associated with task-relevant content, it remains unclear whether MLLMs inadvertently memorize private content that is entirely irrelevant to the training tasks. In this paper, we investigate how randomly generated task-irrelevant private content can become spuriously correlated with downstream objectives due to partial mini-batch training dynamics, thus causing inadvertent memorization. Concretely, we randomly generate task-irrelevant watermarks into VQA fine-tuning images at varying probabilities and propose a novel probing framework to determine whether MLLMs have inadvertently encoded such content. Our experiments reveal that MLLMs exhibit notably different training behaviors in partial mini-batch settings with task-irrelevant watermarks embedded. Furthermore, through layer-wise probing, we demonstrate that MLLMs trigger distinct representational patterns when encountering previously seen task-irrelevant knowledge, even if this knowledge does not influence their output during prompting. Our code is available at https://github.com/illusionhi/ProbingPrivacy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。