发现个性化模型会错误使用视觉记录,提出三重授权机制诊断并修复该问题。
Whose record is this? Diagnosing and authorizing record use in personalized multimodal models

- 构建三重授权条件:主体存在、记录有效、答案支持同时满足才可使用记录。
- 实测多模型在25.75%准确召回下仍有63.69%未经授权使用记录。
- 通过类型预生成授权,将错误使用率从43.63%降至3.06%,显著提升安全性。
情境化视觉个性化可能检索到真实记录却错误应用于错误视觉主体。我们形式化记录可作为条件的授权标准:主体存在(P)、记录边有效性(E)和答案支持(S)必须同时成立。违反即为视觉记忆错配(VMM)。我们构建了包含3,690个案例的诊断套件RecordAuth-Diag,仅改变一张图像-记录边,其余如查询、问题、记录文本和图像集合保持不变。卡片移除与伪标签实验表明失败源于所提供记录。原始数据集上的未授权使用覆盖Qwen、Phi、Gemma系列接口:Gemma-3-4B-IT在25.75%清洁召回下达63.69%本地未授权使用。CoViP保持26.02%未授权率,其Qwen基线为22.49%。类型预生成授权将Qwen卡片暴露率从43.63%降至3.06%,正向召回从86.26%降至60.90%。全条件$P\wedge E\wedge S$验证使用560个局部DAVIS案例:顶级相关性与类型授权释放率分别为28.93%和28.39%,但不安全释放率分别为6.79%和0.89%。33个额外不安全案例中,27例属支持问题,4例为边问题,2例为干净案例,0例为边界案例。因此观察到的增量主要由支持判断主导,而非仅边缘检查。外观仅在主体存在条件下提供$E$证据;经认证的主体标记作为存在性证据的充分性控制实例化缺失的见证。
原文摘要 · Abstract (English)
Contextualized visual personalization can retrieve a true record yet apply it to the wrong visual subject. We formalize when a record may condition an answer as \emph{record authorization}: subject presence ($P$), record-edge validity ($E$), and answer support ($S$) must all hold. We call violations visual memory misbinding (VMM). We construct RecordAuth-Diag, a 3,690-case matched diagnostic suite that changes one image--record edge while holding the query, question, record text, and image multiset fixed. Card removal and nonce relabeling attribute these failures to supplied records. Raw-bank failures span Qwen-, Phi-, and Gemma-family interfaces: Gemma-3-4B-IT reaches 63.69\% local unauthorized use at 25.75\% clean recall. CoViP remains at 26.02\%, versus 22.49\% for its Qwen backbone at similar clean recall. Typed pre-generation authorization reduces Qwen card exposure on RecordAuth-Diag from 43.63\% to 3.06\%, while positive recall changes from 86.26\% to 60.90\%. Full $P\wedge E\wedge S$ validation uses 560 localized DAVIS cases: top-1 relevance and typed authorization have comparable release (28.93\% and 28.39\%) but 6.79\% and 0.89\% unsafe release, respectively. Of the 33 additional unsafe cases removed, 27 are support, 4 edge, 2 clean, and 0 boundary cases. Thus the observed increment is an $E\wedge S$ decision dominated by support, not an edge check alone. Appearance supplies $E$ evidence only conditional on $P$; authenticated subject tokens instantiate the missing presence witness as a sufficiency control. The claims concern the evaluated contracts, not natural prevalence, consent, or visual identity
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。