用搜索诱导攻击测试大模型安全漏洞,发现其易被误导输出有害内容。
SearchAttack: Red-Teaming LLMs against Knowledge-to-Action Threats under Online Web Search
- 通过伪装有害意图生成无害表述,诱使模型检索危险信息。
- 实测显示模型在在线搜索中易被诱导输出恶意指令,且未搜也能被引导出错。
- 适合安全研究人员和大模型开发者评估系统风险边界。
近年来,用户越来越关注大模型在开放、知识密集型任务中的幻觉问题,可靠性差距日益凸显。为此,人们越来越多依赖搜索增强的大模型来缓解此问题。然而,基于大模型的搜索也面临滥用风险。一旦返回内容直接包含可直接使用的有害指令或行动建议,就难以撤销或纠正。为探究大模型在搜索场景下的不安全行为,我们提出 SearchAttack 进行红队测试:(1) 利用密集且看似无害的知识重述有害语义,规避直接上下文解码检测,从而诱发不安全信息检索;(2) 通过引导模型追求奖励,使其合成有害的检索结果。我们还构建了一个新兴的、特定领域的非法活动基准数据集,用于基于搜索的威胁评估,并引入事实核查框架,在离线与在线攻击场景中对危害进行建模与量化。大量实验表明,SearchAttack 在攻击搜索增强型大模型方面具有显著效果。此外,我们发现即使没有网络搜索能力,大模型仍可能因信息寻求的固有行为模式而被引导至有害内容输出。
原文摘要 · Abstract (English)
Recently, people have suffered from LLM hallucination and have become increasingly aware of the reliability gap of LLMs in open and knowledge-intensive tasks. As a result, they have increasingly turned to search-augmented LLMs to mitigate this issue. However, LLM-driven search also becomes an attractive target for misuse. Once the returned content directly contains targeted, ready-to-use harmful instructions or takeaways for users, it becomes difficult to withdraw or undo such exposure. To investigate LLMs' unsafe search behavior issues, we first propose \textbf{\textit{SearchAttack}} for red-teaming, which (1) rephrases harmful semantics via dense and benign knowledge to evade direct in-context decoding, thus eliciting unsafe information retrieval, (2) stress-tests LLMs' reward-chasing bias by steering them to synthesize unsafe retrieved content. We also curate an emergent, domain-specific illicit activity benchmark for search-based threat assessment, and introduce a fact-checking framework to ground and quantify harm in both offline and online attack settings. Extensive experiments are conducted to red-team the search-augmented LLMs for responsible vulnerability assessment. Empirically, SearchAttack demonstrates strong effectiveness in attacking these systems. We also find that LLMs without web search can still be steered into harmful content output due to their information-seeking stereotypical behaviors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。