用强化学习+博弈策略，自动发现大模型漏洞并生成更自然的攻击提示。
Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Experts
- 训练多个专注不同攻击风格的轻量专家，通过强化学习优化。
- 在AdvBench上达到最高成功率（ASR@10），且生成提示更接近人类语言。
- 可诊断模型弱点，揭示哪种攻击方式最易触发不安全响应。
自动化红队测试已成为大规模审计大语言模型部署前安全性的有效方法，但现有方法缺乏在推理阶段针对模型特异性漏洞进行高效适应的能力。本文提出Red-Bandit框架，通过在线自适应识别并利用目标模型在不同攻击风格（如操纵、俚语）下的失效模式。该框架对一组参数高效的LoRA专家进行后训练，每个专家专精于特定攻击风格，采用基于规则的安全模型作为奖励信号，以强化学习优化生成不安全提示的能力。推理时，多臂老虎机策略根据目标模型回复的安全性动态选择攻击专家，平衡探索与利用。Red-Bandit在足够探索条件下，在AdvBench上实现当前最优的攻击成功率（ASR@10），同时生成更具人类可读性的提示（更低困惑度）。此外，其带兵策略可作为诊断工具，揭示哪些攻击风格最易诱发模型产生不安全行为。
原文摘要 · Abstract (English)
Automated red-teaming has emerged as a scalable approach for auditing Large Language Models (LLMs) prior to deployment, yet existing approaches lack mechanisms to efficiently adapt to model-specific vulnerabilities at inference. We introduce Red-Bandit, a red-teaming framework that adapts online to identify and exploit model failure modes under distinct attack styles (e.g., manipulation, slang). Red-Bandit post-trains a set of parameter-efficient LoRA experts, each specialized for a particular attack style, using reinforcement learning that rewards the generation of unsafe prompts via a rule-based safety model. At inference, a multi-armed bandit policy dynamically selects among these attack-style experts based on the target model's response safety, balancing exploration and exploitation. Red-Bandit achieves state-of-the-art results on AdvBench under sufficient exploration (ASR@10), while producing more human-readable prompts (lower perplexity). Moreover, Red-Bandit's bandit policy serves as a diagnostic tool for uncovering model-specific vulnerabilities by indicating which attack styles most effectively elicit unsafe behaviors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。