首个评估视觉语言模型隐私推理能力的基准，揭示60%主流模型可高精度还原个人身份。
MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models
- 构建双语多模态数据集，模拟个体信息链式推理场景
- 60%主流模型在9项任务中实现最高80%的隐私推理准确率
- 适合关注AI隐私风险的研究者与安全评估人员
现代视觉语言模型（VLMs）通过分层思维链推理，可能将分散的多模态数据关联至可识别个体，构成严重个人隐私风险。然而现有隐私评测基准结构不足，仅评估隐私感知而忽略更关键的隐私推理能力——即模型整合分布信息构建个体画像的能力。为此，我们提出MultiPriv，首个系统评估VLMs个体级隐私推理能力的基准。引入隐私感知与推理（PPR）框架，构建含合成个体身份的双语多模态数据集，其中人脸、姓名等标识符与敏感属性关联。该设计支持九类挑战性任务，涵盖属性检测、跨图像重识别及链式推理。对超过50个开源与商用VLMs进行大规模评估，在受控环境下，60%广泛使用的VLMs可实现最高80%准确率的个体级隐私推理，表明个人隐私存在显著潜在威胁。基准已公开于https://github.com/CyberChangAn/MultiPriv-PII。
原文摘要 · Abstract (English)
Modern Vision-Language Models (VLMs) pose significant individual-level privacy risks by linking fragmented multimodal data to identifiable individuals through hierarchical chain-of-thought reasoning. However, existing privacy benchmarks remain structurally insufficient for this threat, as they primarily evaluate privacy perception while failing to address the more critical risk of privacy reasoning: a VLM's ability to infer and link distributed information to construct individual profiles. To address this gap, we propose MultiPriv, the first benchmark designed to systematically evaluate individual-level privacy reasoning in VLMs. We introduce the Privacy Perception and Reasoning (PPR) framework and construct a bilingual multimodal dataset with synthetic individual profiles, where identifiers, such as faces and names, are linked to sensitive attributes. This design enables nine challenging tasks spanning attribute detection, cross-image re-identification, and chained inference. We conduct a large-scale evaluation of over 50 open-source and commercial VLMs. In our controlled benchmark, 60% of widely used VLMs can perform individual-level privacy reasoning with up to 80% accuracy, suggesting a significant potential threat to personal privacy. The benchmark is available at https://github.com/CyberChangAn/MultiPriv-PII.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。