arXiv:2506.03179cs.CVcs.AI2025-06NeurIPS被引 18

首个针对视频理解大模型的成员推理攻击方法,可检测训练数据中的敏感视频。

Vid-SME: Membership Inference Attacks against Large Video Understanding Models

  • 利用视频帧时序反转与模型置信度差异,计算沙玛-米塔尔熵差异。
  • 在多个自训练和开源视频大模型上实现高精度成员推断,低误报率下真阳性率显著提升。
  • 适合关注视频模型隐私安全的研究者与开发者使用。

多模态大语言模型(MLLMs)在处理复杂多模态任务方面表现出色,并被广泛应用于视频理解。然而,其快速发展带来了严重数据隐私问题,尤其是训练数据中可能包含个人录像、监控视频等敏感内容,而这些内容是否被不当使用仍难以识别。尽管文本与图像领域的成员推理攻击(MIAs)已有进展,但现有方法无法有效迁移至视频领域,主要因采样更多帧时扩展性差,且在低假阳性率下真阳性率极低,根源在于未能捕捉视频帧的固有时序特性,也未考虑帧数变化对模型行为的影响。为此,本文提出 Vid-SME,首个专为视频理解大模型(VULLMs)设计的成员推理方法。Vid-SME 利用模型输出置信度,结合自适应参数化,计算视频输入的沙玛-米塔尔熵(SME)。通过对比自然视频与时间反转视频的 SME 差异,生成稳健的成员得分,以判断某视频是否属于模型训练集。在多个自训练及开源 VULLM 上的实验表明,Vid-SME 具有强大有效性。

原文摘要 · Abstract (English)

Multimodal large language models (MLLMs) demonstrate remarkable capabilities in handling complex multimodal tasks and are increasingly adopted in video understanding applications. However, their rapid advancement raises serious data privacy concerns, particularly given the potential inclusion of sensitive video content, such as personal recordings and surveillance footage, in their training datasets. Determining improperly used videos during training remains a critical and unresolved challenge. Despite considerable progress on membership inference attacks (MIAs) for text and image data in MLLMs, existing methods fail to generalize effectively to the video domain. These methods suffer from poor scalability as more frames are sampled and generally achieve negligible true positive rates at low false positive rates (TPR@Low FPR), mainly due to their failure to capture the inherent temporal variations of video frames and to account for model behavior differences as the number of frames varies. To address these challenges, we introduce Vid-SME, the first membership inference method tailored for video data used in video understanding LLMs (VULLMs). Vid-SME leverages the confidence of model output and integrates adaptive parameterization to compute Sharma-Mittal entropy (SME) for video inputs. By leveraging the SME difference between natural and temporally-reversed video frames, Vid-SME derives robust membership scores to determine whether a given video is part of the model's training set. Experiments on various self-trained and open-sourced VULLMs demonstrate the strong effectiveness of Vid-SME.

视频理解隐私安全成员推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。