自动复现文本到图像越狱攻击,让实验结果可比可重用。
PixJail: Self-Evolving Paper-to-Pipeline Reproduction for Text-to-Image Jailbreak Evaluation

- 构建可自演化攻击管道的智能代理框架,统一复现流程。
- 复现11种主流越狱方法,平均误差仅2.1%,中位数为0%。
- 适合安全研究者与模型评估团队,减少重复劳动。
随着文本到图像(T2I)越狱技术快速演进,现有基准和复现流程难以跟上。更重要的是,T2I越狱评估不是单一提示测试,而是由提示转换、图像生成、安全过滤和多模态判断等多个阶段构成的全流程问题,导致论文间结果难以可靠复现与公平比较。为此,我们提出PixJail:一种从论文到可运行攻击管道的自演化智能代理框架,能基于论文与可选代码快速构建特定攻击模块与评估流水线,并在统一协议下忠实复现原始实验结果。系统维护记忆库,保存论文摘要、攻击演化模式、可复用模板、失败案例与版本化产物,支持未来复现时复用历史经验。我们成功复现了11种代表性T2I越狱方法,包括有无代码公开的论文,在原设置下平均误差仅2.1%,中位数为0%。希望PixJail能成为未来T2I越狱复现与评估的统一基础,显著降低人工成本。
原文摘要 · Abstract (English)
As Text-to-Image (T2I) jailbreak techniques evolve rapidly, existing benchmarks and reproduction workflows often struggle to keep pace. More importantly, T2I jailbreak evaluation is not a single prompt-level test, but a pipeline-level problem shaped by multiple stages, including prompt transformation, image generation, safety filtering, and multimodal judging. This makes results across papers difficult to reliably reproduce and fairly compare. To bridge this gap, we propose PixJail, a self-evolving paper-to-pipeline agent framework for reproducible T2I jailbreak evaluation. Given a T2I jailbreak paper and optional reference code, PixJail rapidly constructs a paper-specific attack module and a runnable evaluation pipeline under a unified contract, while faithfully reproducing the original experimental results. PixJail further maintains a memory bank that stores paper digests, attack evolution patterns, reusable templates, failure cases, and versioned artifacts, enabling future reproduction efforts to reuse prior experience. We reproduce eleven representative T2I jailbreak methods, including both code-available and code-unavailable papers. Under their original settings, our framework accurately recovers prior results with minimal error (2.1\% average, 0\% median). We hope that PixJail can serve as a unified foundation for future T2I jailbreak reproduction and evaluation, significantly reducing manual effort.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。