用智能体生成更真实工业缺陷图像,提升检测模型性能。
AnomalyAgent: Agentic Industrial Anomaly Synthesis via Tool-Augmented Reinforcement Learning

- 构建带工具的智能体,支持自我反思与迭代优化
- 在MVTec-AD上实现2.10/0.33的异常生成指标
- 适合需要高质量合成数据的工业缺陷检测研究者
工业异常生成是缓解异常检测中数据稀缺问题的关键方法。现有方法多依赖单步生成机制,缺乏复杂推理与迭代优化能力,难以生成高语义真实性的异常样本。本文提出AnomalyAgent,一种具备自我反思、知识检索和迭代优化能力的异常生成智能体,旨在生成逼真且多样化的异常。AnomalyAgent配备五种工具:提示生成(PG)、图像生成(IG)、质量评估(QE)、知识检索(KR)和掩码生成(MG),实现闭环优化。为提升决策与自省能力,我们基于真实异常图像构建结构化轨迹,并设计两阶段训练框架:监督微调后接强化学习。该过程由三部分奖励机制驱动:(1)任务奖励,监督生成异常的质量与位置合理性;(2)反思奖励,训练模型改进生成提示;(3)行为奖励,确保遵循轨迹。在MVTec-AD数据集上,AnomalyAgent的异常生成指标IS/IC-L达2.10/0.33,使用ResNet34分类准确率达57.0%,采用简单UNet在图像/像素级的平均精度分别为99.3%/74.2%,超越所有零样本最先进方法。代码与数据将公开。
原文摘要 · Abstract (English)
Industrial anomaly generation is a crucial method for alleviating the data scarcity problem in anomaly detection tasks. Most existing anomaly synthesis methods rely on single-step generation mechanisms, lacking complex reasoning and iterative optimization capabilities, making it difficult to generate anomaly samples with high semantic realism. We propose AnomalyAgent, an anomaly synthesis agent with self-reflection, knowledge retrieval, and iterative refinement capabilities, aiming to generate realistic and diverse anomalies. Specifically, AnomalyAgent is equipped with five tools: Prompt Generation (PG), Image Generation (IG), Quality Evaluation (QE), Knowledge Retrieval (KR), and Mask Generation (MG), enabling closed-loop optimization. To improve decision-making and self-reflection, we construct structured trajectories from real anomaly images and design a two-stage training framework: supervised fine-tuning followed by reinforcement learning. This process is driven by a three-part reward mechanism: (1) task rewards to supervise the quality and location rationality of generated anomalies; (2) reflection rewards to train the model's ability to improve anomaly synthesis prompt; (3) behavioral rewards to ensure adherence to the trajectory. On the MVTec-AD dataset, AnomalyAgent achieves IS/IC-L of 2.10/0.33 for anomaly generation, 57.0% classification accuracy using ResNet34, and 99.3%/74.2% AP at the image/pixel level using a simple UNet, surpassing all zero-shot SOTA methods. The code and data will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。