arXiv:2410.12586cs.CL2024-10NAACL被引 10

提出可逆向删除大模型上下文编辑内容的新方法,提升模型安全性与透明度。

How to Make LLMs Forget: On Reversing In-Context Knowledge Edits

  • 通过分析前10个词的概率分布,实现对编辑内容的高精度检测。
  • 设计连续与离散反向标记,恢复超过80%原始输出,且不影响未编辑输入。
  • 适用于封闭模型环境,为对抗恶意编辑提供可操作的防御方案。

上下文知识编辑(IKE)可在不修改参数、零成本的情况下高效调整大语言模型(LLM)输出,但可能被滥用以隐匿方式注入错误信息或不当内容,尤其在无法查看最终提示的封装API中。为此,本文研究了IKE编辑的检测与逆转。首先,仅利用下一个词的前10个最高输出概率,即可在黑盒环境下实现超过80%的F1准确率检测。其次,提出新任务:使用特制反向标记逆转IKE编辑。探索连续与离散反向标记,在多个LLM上实现超过80%的原始输出恢复率。连续反向标记表现尤为出色,对未编辑输入影响极小。通过分析输出分布、注意力模式与词元排序,揭示IKE对模型的影响机制及反向标记的缓解作用。本工作显著增强了LLM对上下文编辑滥用的抗性,提升了其透明度与可信度。

原文摘要 · Abstract (English)

In-context knowledge editing (IKE) enables efficient modification of large language model (LLM) outputs without parameter changes and at zero-cost. However, it can be misused to manipulate responses opaquely, e.g., insert misinformation or offensive content. Such malicious interventions could be incorporated into high-level wrapped APIs where the final input prompt is not shown to end-users. To address this issue, we investigate the detection and reversal of IKE-edits. First, we demonstrate that IKE-edits can be detected with high accuracy (F1 > 80\%) using only the top-10 output probabilities of the next token, even in a black-box setting, e.g. proprietary LLMs with limited output information. Further, we introduce the novel task of reversing IKE-edits using specially tuned reversal tokens. We explore using both continuous and discrete reversal tokens, achieving over 80\% accuracy in recovering original, unedited outputs across multiple LLMs. Our continuous reversal tokens prove particularly effective, with minimal impact on unedited prompts. Through analysis of output distributions, attention patterns, and token rankings, we provide insights into IKE's effects on LLMs and how reversal tokens mitigate them. This work represents a significant step towards enhancing LLM resilience against potential misuse of in-context editing, improving their transparency and trustworthiness.

大模型安全知识编辑反向恢复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。