arXiv:2603.12598cs.CV2026-03中稿 · ECCV

通过神经元级编辑提升大模型隐私防护,拒绝敏感请求更有效且不损害原功能。

Neural Gate: Mitigating Privacy Risks in LVLMs via Neuron-Level Gradient Gating

  • 定位模型中与隐私相关的神经元,精准调整参数以增强防护。
  • 在未见隐私问题上拒答率提升至92%,同时保持原任务性能稳定。
  • 适合关注模型安全、隐私保护的研究者与应用开发者使用。

大型视觉语言模型(LVLMs)在金融、医疗等关键领域展现出巨大潜力,但其部署也带来严重隐私风险。恶意用户可能利用模型提取敏感信息,而现有方法在应对未见隐私指令时效果不佳,且常损害模型在常规任务上的表现。为此,我们提出Neural Gate,一种基于神经元级别的模型编辑方法,通过学习特征向量识别模型中与隐私相关概念的神经元,并精准引导参数更新。该方法显著提升模型对隐私相关提问的拒绝率,关键在于能有效扩展到训练中未见过的新敏感查询。在MiniGPT和LLaVA上的实验表明,该方法在提升隐私保护的同时,维持了模型原有能力。代码已开源:https://github.com/Xiangkui-Cao/Neural-Gate。

原文摘要 · Abstract (English)

Large Vision-Language Models (LVLMs) have shown remarkable potential across a wide array of vision-language tasks, leading to their adoption in critical domains such as finance and healthcare. However, their growing deployment also introduces significant security and privacy risks. Malicious actors could potentially exploit these models to extract sensitive information, highlighting a critical vulnerability. Recent studies show that LVLMs often fail to consistently refuse instructions designed to compromise user privacy. While existing work on privacy protection has made meaningful progress in preventing the leakage of sensitive data, they are constrained by limitations in both generalization and non-destructiveness. They often struggle to robustly handle unseen privacy-related queries and may inadvertently degrade a model's performance on standard tasks. To address these challenges, we introduce Neural Gate, a novel method for mitigating privacy risks through neuron-level model editing. Our method improves a model's privacy safeguards by increasing its rate of refusal for privacy-related questions, crucially extending this protective behavior to novel sensitive queries not encountered during the editing process. Neural Gate operates by learning a feature vector to identify neurons associated with privacy-related concepts within the model's representation of a subject. This localization then precisely guides the update of model parameters. Through comprehensive experiments on MiniGPT and LLaVA, we demonstrate that our method significantly boosts the model's privacy protection while preserving its original utility. The code is available at https://github.com/Xiangkui-Cao/Neural-Gate.

隐私保护大模型安全神经元编辑视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。