用视觉语言描述词解释图像隐私，让判断结果更透明可信。
Image-guided topic modeling for interpretable privacy classification
- 通过多模态对齐生成与图像相关的自然语言描述词
- 隐私分类准确率比现有可解释方法高5个百分点
- 适合需要透明决策的隐私保护场景
预测并以人类可理解的方式解释图像中包含的私密信息是一项复杂且依赖语境的任务，甚至对大型语言模型也极具挑战。为提升隐私判断的可解释性，我们提出基于一组自然语言内容描述词来预测图像隐私。这些描述词关联有反映人们感知的隐私评分。我们采用新颖的图像引导主题建模（ITM）方法生成描述词，该方法通过多模态对齐融合视觉信息和视觉-语言模型提供的文本描述。利用ITM生成的描述词训练隐私分类器Priv×ITM，其决策过程天然可解释。实验表明，Priv×ITM分类器在准确率上比基准可解释方法高出5个百分点，且性能接近当前非可解释的最先进模型。
原文摘要 · Abstract (English)
Predicting and explaining the private information contained in an image in human-understandable terms is a complex and contextual task. This task is challenging even for large language models. To facilitate the understanding of privacy decisions, we propose to predict image privacy based on a set of natural language content descriptors. These content descriptors are associated with privacy scores that reflect how people perceive image content. We generate descriptors with our novel Image-guided Topic Modeling (ITM) approach. ITM leverages, via multimodality alignment, both vision information and image textual descriptions from a vision language model. We use the ITM-generated descriptors to learn a privacy predictor, Priv$\times$ITM, whose decisions are interpretable by design. Our Priv$\times$ITM classifier outperforms the reference interpretable method by 5 percentage points in accuracy and performs comparably to the current non-interpretable state-of-the-art model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。