arXiv:2605.08800cs.CVcs.AI2026-05

针对视觉语言模型的个性化部分遗忘,构建真实世界基准并提出边界感知优化方法。

PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models

论文配图:PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models
图 1 · 摘自论文原文
  • 构建24K样本的真实世界基准,支持三类渐进式遗忘任务。
  • 发现现有方法在保留非目标事实时存在显著遗忘-保留权衡。
  • 提出边界感知优化,精准控制同一主体内知识的遗忘范围。

多模态大语言模型(MLLM)在预训练过程中可能记忆敏感的跨模态信息。现有MLLM遗忘评估基准依赖合成知识注入或完整主体删除,无法反映真实场景下的个性化遗忘需求,即需要细粒度的事实控制。本文提出PPU-Bench,一个无需微调的真实世界个性化部分遗忘基准。该基准基于500位公众人物的已有知识,生成24,000个跨模态与单模态样本,涵盖完全、选择性与个性化三种渐进挑战场景。评估方法是否能在移除目标知识的同时,保持非目标事实、模型性能与跨模态一致性。实验表明,完全遗忘常抑制视觉身份而非事实知识;选择性与个性化遗忘暴露了主体内部事实边界的显著遗忘-保留权衡。跨图像与提示攻击的鲁棒性分析揭示不同遗忘设置下的独特漏洞。受此启发,我们提出边界感知优化(BAO),显式建模主体内遗忘-保留边界。在两种代表性方法上的实验验证其能有效约束主体内事实边界。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) may memorize sensitive cross-modal information during pretraining. However, existing MLLM unlearning benchmarks rely on synthetic knowledge injection or complete subject-level deletion, which fail to capture realistic, personalized deletion requests that require fine-grained factual control. In this paper, we introduce PPU-Bench, a real-world and fine-tuning-free benchmark for personalized partial unlearning in MLLMs. PPU-Bench contains 24K multimodal and unimodal samples derived from pre-existing knowledge of 500 public figures under three progressively challenging settings: Complete, Selective, and Personalized unlearning. The benchmark evaluates whether methods can remove target knowledge while preserving non-target facts, model utility, and cross-modal consistency. Extensive experiments show that Complete Unlearning often suppresses visual identity rather than factual knowledge, while Selective and Personalized Unlearning expose significant forget--retain trade-offs and challenges in intra-subject factual boundaries. Robustness analysis under cross-image and prompt-based attacks reveals distinct vulnerabilities across different unlearning settings. Motivated by these findings, we propose Boundary-Aware Optimization (BAO), which explicitly models intra-subject forget-retain boundaries. Experimental results on two representative methods demonstrate that BAO can effectively enforce intra-subject factual boundaries.

模型遗忘多模态边界控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。