用精炼推理蒸馏出高效红队模型，提升大模型代理的安全测试覆盖与攻击成功率。
SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning
- 分两步生成多样测试用例并迭代优化对抗攻击路径。
- 测试覆盖提升2-2.5倍，攻击成功率翻倍超越671B大模型。
- 适合安全评估、模型审计及研发团队做红队测试。
LLM代理的规划与工具调用能力带来了新的安全风险，因此全面的红队测试系统对发现漏洞和保障安全部署至关重要。我们提出SIRAJ：一个适用于任意黑盒LLM代理的通用红队框架。采用动态双阶段流程，先基于代理定义生成覆盖多种风险结果、工具调用轨迹和风险来源的多样化种子测试用例；再基于前序执行轨迹迭代构建并优化模型驱动的对抗攻击。为降低红队成本，我们提出模型蒸馏方法，利用教师模型的结构化推理形式训练出效果相当但更小的模型。在多种评估代理设置下，我们的种子生成方法使风险结果和工具调用轨迹覆盖率提升2–2.5倍；蒸馏后的8B红队模型将攻击成功率提升100%，超越671B的Deepseek-R1模型。消融实验与分析验证了迭代框架、结构化推理及红队模型泛化能力的有效性。
原文摘要 · Abstract (English)
The ability of LLM agents to plan and invoke tools exposes them to new safety risks, making a comprehensive red-teaming system crucial for discovering vulnerabilities and ensuring their safe deployment. We present SIRAJ: a generic red-teaming framework for arbitrary black-box LLM agents. We employ a dynamic two-step process that starts with an agent definition and generates diverse seed test cases that cover various risk outcomes, tool-use trajectories, and risk sources. Then, it iteratively constructs and refines model-based adversarial attacks based on the execution trajectories of former attempts. To optimize the red-teaming cost, we present a model distillation approach that leverages structured forms of a teacher model's reasoning to train smaller models that are equally effective. Across diverse evaluation agent settings, our seed test case generation approach yields 2 -- 2.5x boost to the coverage of risk outcomes and tool-calling trajectories. Our distilled 8B red-teamer model improves attack success rate by 100%, surpassing the 671B Deepseek-R1 model. Our ablations and analyses validate the effectiveness of the iterative framework, structured reasoning, and the generalization of our red-teamer models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。