用进化算法自动生成毒害性提示，测试大模型安全漏洞。
ToxSearch: Evolving Prompts for Toxicity Search in Large Language Models
- 通过同步演化循环，结合多种语言操作符生成攻击性提示。
- 小扰动即可诱发毒性，跨模型迁移后毒性约减半，小模型更抗攻击。
- 适合安全研究人员做红队测试，验证模型泛化防御能力。
大型语言模型即使经过安全对齐，仍可能被对抗性提示诱导输出有毒内容。我们提出 ToxSearch，一个黑盒进化框架，通过同步稳态循环演化提示以测试模型安全性。系统采用多样化操作符，包括词汇替换、否定、回译、改写及两种语义交叉操作，并由内容审核代理提供适应度指导。操作符分析显示：词汇替换在收益与方差间表现最优，语义相似性交叉为高精度低效插入器，全局重写则方差大且拒绝成本高。使用在 LLaMA 3.1 8B 上演化的精英提示，观察到实际有意义但衰减的跨模型迁移效果：多数目标模型毒性减半，较小的 LLaMA 3.2 变体表现出最强抵抗，部分跨架构模型仍保留较高毒性。结果表明，微小可控扰动是系统性红队测试的有效工具，防御策略应考虑对抗提示的跨模型复用，而不仅聚焦单模型加固。
原文摘要 · Abstract (English)
Large Language Models remain vulnerable to adversarial prompts that elicit toxic content even after safety alignment. We present ToxSearch, a black-box evolutionary framework that tests model safety by evolving prompts in a synchronous steady-state loop. The system employs a diverse set of operators, including lexical substitutions, negation, back-translation, paraphrasing, and two semantic crossover operators, while a moderation oracle provides fitness guidance. Operator-level analysis shows heterogeneous behavior: lexical substitutions offer the best yield-variance trade-off, semantic-similarity crossover acts as a precise low-throughput inserter, and global rewrites exhibit high variance with elevated refusal costs. Using elite prompts evolved on LLaMA 3.1 8B, we observe practically meaningful but attenuated cross-model transfer, with toxicity roughly halving on most targets, smaller LLaMA 3.2 variants showing the strongest resistance, and some cross-architecture models retaining higher toxicity. These results suggest that small, controllable perturbations are effective vehicles for systematic red-teaming and that defenses should anticipate cross-model reuse of adversarial prompts rather than focusing only on single-model hardening.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。