提出新基准ReMem，解决视觉语言模型记忆失效问题
Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks
- 设计多跳多图记忆测试，确保模型先学会再删
- 实测发现现有基准因记不住导致评估失真
- 适合研究模型遗忘机制与隐私保护的学者
大型视觉语言模型虽强大，却可能无意中记忆敏感个人信息。现有遗忘评估基准使用虚构身份，却忽视了关键的第一阶段失败：模型未能有效记忆目标信息，导致后续遗忘评估不可靠。我们诊断出记忆不足与多跳推理困境是根源，提出ReMem——一个可靠的多跳多图像记忆基准。通过合理数据扩展、关注推理的问答对和多样视觉场景，确保模型具备扎实的记忆基础。同时提出新颖的曝光度指标，量化模型内部概率分布中信息擦除的深度。大量实验表明，ReMem为诊断LVLM的学习与遗忘行为提供了严谨可信的框架。
原文摘要 · Abstract (English)
While Large Vision-Language Models (LVLMs) offer powerful capabilities, they pose privacy risks by unintentionally memorizing sensitive personal information. Current unlearning benchmarks attempt to mitigate this using fictitious identities but overlook a critical stage 1 failure: models fail to effectively memorize target information initially, rendering subsequent unlearning evaluations unreliable. Diagnosing under-memorization and the multi-hop curse as root causes, we introduce ReMem, a Reliable Multi-hop and Multi-image Memorization Benchmark. ReMem ensures robust foundational learning through principled data scaling, reasoning-aware QA pairs, and diverse visual contexts. Additionally, we propose a novel Exposure metric to quantify the depth of information erasure from the model's internal probability distribution. Extensive experiments demonstrate that ReMem provides a rigorous and trustworthy framework for diagnosing both learning and unlearning behaviors in LVLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。