arXiv:2602.20543cs.CV2026-02

用多智能体系统提升药厂菌落检测自动化,准确率超99%。

Beyond Human Performance: A Vision-Language Multi-Agent Approach for Quality Control in Pharmaceutical Manufacturing

  • 融合深度学习与视觉语言模型,分步判断培养皿有效性并独立计数
  • 实现99%检出率、2%假阳性、0.6%假阴性,异常样本识别能力显著提升
  • 可自动对接数据库,减少50%以上人工复核,适合制药质量控制场景

菌落形成单位(CFU)检测在制药生产中至关重要,是环境监测的核心环节,关乎是否符合严格的质量标准。人工计数耗时且易错,而深度学习虽准确但仍受样本质量与伪影影响。基于此前的CNN框架,我们评估了YOLOv5、YOLOv7和YOLOv8,但最高仅达97.08%准确率,不满足药企要求。采用自定义Detectron2模型,在超过5万张培养皿图像数据集上训练,实现99%检出率、2%假阳性、0.6%假阴性。然而该模型在污染板、塑料伪影或成像模糊等异常情况下降效。为此,我们构建了结合深度学习与视觉语言模型(VLM)的多智能体系统:VLM先判断培养皿是否有效;有效样本由DL与VLM独立计数,若结果差异小于5%,则自动录入Postgres与SAP;否则转交专家复核,反馈用于持续迭代优化。初期基于深度学习的自动化使疫苗生产站点的人工验证减少50%,引入VLM后提升至85%,带来显著运营效益。该系统提供可扩展、可审计、合规的微生物质控方案,推动生物制药自动化发展。

原文摘要 · Abstract (English)

Colony-forming unit (CFU) detection is critical in pharmaceutical manufacturing, serving as a key component of Environmental Monitoring programs and ensuring compliance with stringent quality standards. Manual counting is labor-intensive and error-prone, while deep learning (DL) approaches, though accurate, remain vulnerable to sample quality variations and artifacts. Building on our earlier CNN-based framework (Beznik et al., 2020), we evaluated YOLOv5, YOLOv7, and YOLOv8 for CFU detection; however, these achieved only 97.08 percent accuracy, insufficient for pharmaceutical-grade requirements. A custom Detectron2 model trained on GSK's dataset of over 50,000 Petri dish images achieved 99 percent detection rate with 2 percent false positives and 0.6 percent false negatives. Despite high validation accuracy, Detectron2 performance degrades on outlier cases including contaminated plates, plastic artifacts, or poor optical clarity. To address this, we developed a multi-agent framework combining DL with vision-language models (VLMs). The VLM agent first classifies plates as valid or invalid. For valid samples, both DL and VLM agents independently estimate colony counts. When predictions align within 5 percent, results are automatically recorded in Postgres and SAP; otherwise, samples are routed for expert review. Expert feedback enables continuous retraining and self-improvement. Initial DL-based automation reduced human verification by 50 percent across vaccine manufacturing sites. With VLM integration, this increased to 85 percent, delivering significant operational savings. The proposed system provides a scalable, auditable, and regulation-ready solution for microbiological quality control, advancing automation in biopharmaceutical production.

质量控制视觉语言模型多智能体制药自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。