让大模型自己说出信心程度，用自然语言做攻击优化。
Black-box Optimization of LLM Outputs by Asking for Directions
- 诱导大模型用自然语言表达内部置信度，实现黑盒攻击
- 在视觉-语言模型、越狱和提示注入中均成功生成恶意输入
- 模型越大越准，能力提升反而加剧安全风险
我们提出一种新方法，通过利用大语言模型（LLM）在自然语言中表达置信度的能力，对黑盒大模型进行攻击。现有黑盒攻击通常依赖连续输出如logits或置信分数（实际很少可用），或依赖其他模型的代理信号。我们展示如何通过提示使LLM以足够校准的方式表达内部置信度，从而实现有效的对抗优化。该方法适用于三种攻击场景：视觉-语言模型的对抗样本、越狱和提示注入。攻击成功生成针对仅暴露文本输出系统的恶意输入，显著扩大了部署型大模型的攻击面。此外，我们发现更大、更好的模型在表达置信度时具有更优的校准性，形成一个令人担忧的安全悖论——模型能力提升直接增强了其脆弱性。代码已开源。
原文摘要 · Abstract (English)
We present a novel approach for attacking black-box large language models (LLMs) by exploiting their ability to express confidence in natural language. Existing black-box attacks require either access to continuous model outputs like logits or confidence scores (which are rarely available in practice), or rely on proxy signals from other models. Instead, we demonstrate how to prompt LLMs to express their internal confidence in a way that is sufficiently calibrated to enable effective adversarial optimization. We apply our general method to three attack scenarios: adversarial examples for vision-LLMs, jailbreaks and prompt injections. Our attacks successfully generate malicious inputs against systems that only expose textual outputs, thereby dramatically expanding the attack surface for deployed LLMs. We further find that better and larger models exhibit superior calibration when expressing confidence, creating a concerning security paradox where model capability improvements directly enhance vulnerability. Our code is available at this [link](https://github.com/zj-jayzhang/black_box_llm_optimization).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。