用流程挖掘分析大模型对抗攻击的动态过程，发现防御机制差异。
Beyond Pass/Fail: Using Process Mining to Understand How LLMs Resist (and Fail) Red Team Attacks
- 将流程挖掘应用于红队攻击日志，揭示模型拒绝与突破的动态路径。
- GPT-OSS近似吸收态拒答，而Llama存在多个可突破的逃逸路径。
- 不同攻击策略对模型效果不对称，突破时间相差一个数量级。
标准的AI红队评估将对抗攻击简化为单一二元结果——攻击成功率（ASR），忽略了模型抵抗或屈服于攻击的时序结构。本文提出将流程挖掘（process mining）应用于红队攻击轨迹分析。在受控实验中，将60个HarmBench提示分别针对GPT-OSS 120B和Llama 3.3 70B模型，使用10种提示变异策略，每条提示最多尝试110次，共生成8,575个评分事件。从中提取直接跟随图（DFGs）与状态转移矩阵，揭示了仅靠ASR无法捕捉的结构性防御特征：GPT-OSS表现出近乎吸收态的拒绝状态，而Llama则存在多个脆弱的逃逸路径。进一步发现，变异器对不同模型的效果具有不对称性，且突破时间分布相差一个数量级。
原文摘要 · Abstract (English)
Standard AI red teaming evaluations reduce adversarial campaigns to a single binary outcome, attack success rate (ASR), not taking into account the sequential structure of how models resist or yield to attacks. We propose applying process mining, a discipline for discovering and analyzing process models from event logs, to red teaming traces. We conduct a controlled experiment pitting 60 HarmBench prompts against two LLMs, GPT-OSS 120B and Llama 3.3 70B, using 10 prompt mutation strategies over up to 110 attempts per prompt. From the resulting 8,575 scored events we extract Directly-Follows Graphs (DFGs) and state transition matrices that reveal structurally distinct defense profiles invisible to ASR alone: GPT-OSS exhibits a near-absorbing refusal state, while Llama presents multiple porous escape routes from refusal to getting successfully jailbroken. We further show that mutator effectiveness is asymmetric across models and that time-to-jailbreak distributions differ by an order of magnitude.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。