自动化测试大模型搜索代理的安全漏洞，发现其易受误导且防御有限。
SafeSearch: Automated Red-Teaming of LLM-Based Search Agents
- 构建可扩展的红队测试框架SafeSearch，模拟真实攻击场景。
- 在17个大模型上测试，最高攻击成功率超90.5%（GPT-4.1-mini）。
- 适合安全研究人员和模型开发者用于评估与改进搜索代理安全。
搜索代理将大语言模型与互联网连接，以获取更广泛和实时的信息，但也引入了新的安全威胁：不可靠的搜索结果可能误导代理生成不安全输出。现实案例及我们两次实地观察表明此类问题确实在实践中发生。为系统研究该威胁，我们提出SafeSearch——一个可扩展、低成本且轻量的自动化红队框架，支持沙箱环境下的安全评估。通过该框架，我们生成了300个涵盖五类风险（如虚假信息、提示注入）的测试用例，并在17个代表性大模型上评估了三个搜索代理原型。结果揭示基于大模型的搜索代理存在显著漏洞，其中GPT-4.1-mini在搜索工作流设置下的最高攻击成功率（ASR）达90.5%。此外，我们发现常见防御措施如提醒提示效果有限。总体而言，SafeSearch为衡量和提升大模型搜索代理的安全性提供了实用路径。
原文摘要 · Abstract (English)
Search agents connect LLMs to the Internet, enabling them to access broader and more up-to-date information. However, this also introduces a new threat surface: unreliable search results can mislead agents into producing unsafe outputs. Real-world incidents and our two in-the-wild observations show that such failures can occur in practice. To study this threat systematically, we propose SafeSearch, an automated red-teaming framework that is scalable, cost-efficient, and lightweight, enabling sandboxed safety evaluation of search agents. Using this, we generate 300 test cases spanning five risk categories (e.g., misinformation and prompt injection) and evaluate three search agent scaffolds across 17 representative LLMs. Our results reveal substantial vulnerabilities in LLM-based search agents, with the highest ASR reaching 90.5% for GPT-4.1-mini in a search-workflow setting. Moreover, we find that common defenses, such as reminder prompting, offer limited protection. Overall, SafeSearch provides a practical way to measure and improve the safety of LLM-based search agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。