用信息论量化大模型攻击中每轮泄露的比特数,给出安全披露的理论上限。
Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMs
- 以互信息衡量每轮响应泄露的比特数,建立攻击成本的理论下界。
- 仅暴露答案令牌需千次查询,加入逻辑值可降至百次,完整推理过程仅需几十次。
- 为模型透明与安全之间的权衡提供首个可量化的评估标准,适合安全审计者使用。
针对大语言模型(LLMs)的安全威胁,恶意用户通过对抗性指令试图推断目标属性 $T$,如触发有害响应的二元标志或遗忘后信息恢复程度。模型在回复中会泄露可观测信号 $Z$,包含答案标记、推理过程标记或逻辑值。然而,泄露信息量一直缺乏定量评估,导致审计者无据可依,防御者难以权衡透明度与风险。本文提出信息论框架,将 $Z$ 与 $T$ 间的互信息 $I(Z;T)$ 定义为每轮查询泄露的比特数。理论表明:实现误差 $\varepsilon$ 至少需要 $\log(1/\varepsilon)/I(Z;T)$ 次查询,其代价随泄露率倒数线性增长,对精度仅对数依赖。因此,小幅增加披露即可使攻击成本从二次级降至对数级。七种模型在系统提示泄露、越狱攻击和重学习攻击上的实验验证了该理论:仅暴露答案标记需约一千次查询;加入逻辑值可降至约一百次;揭示完整思考过程则缩至几十次。本研究首次提供了部署大模型时透明与安全平衡的理论基准。
原文摘要 · Abstract (English)
Adversarial attacks by malicious users that threaten the safety of large language models (LLMs) can be viewed as attempts to infer a target property $T$ that is unknown when an instruction is issued, and becomes knowable only after the model's reply is observed. Examples of target properties $T$ include the binary flag that triggers an LLM's harmful response or rejection, and the degree to which information deleted by unlearning can be restored, both elicited via adversarial instructions. The LLM reveals an \emph{observable signal} $Z$ that potentially leaks hints for attacking through a response containing answer tokens, thinking process tokens, or logits. Yet the scale of information leaked remains anecdotal, leaving auditors without principled guidance and defenders blind to the transparency--risk trade-off. We fill this gap with an information-theoretic framework that computes how much information can be safely disclosed, and enables auditors to gauge how close their methods come to the fundamental limit. Treating the mutual information $I(Z;T)$ between the observation $Z$ and the target property $T$ as the leaked bits per query, we show that achieving error $\varepsilon$ requires at least $\log(1/\varepsilon)/I(Z;T)$ queries, scaling linearly with the inverse leak rate and only logarithmically with the desired accuracy. Thus, even a modest increase in disclosure collapses the attack cost from quadratic to logarithmic in terms of the desired accuracy. Experiments on seven LLMs across system-prompt leakage, jailbreak, and relearning attacks corroborate the theory: exposing answer tokens alone requires about a thousand queries; adding logits cuts this to about a hundred; and revealing the full thinking process trims it to a few dozen. Our results provide the first principled yardstick for balancing transparency and security when deploying LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。