用大模型生成绝缘子缺陷图像,解决真实数据少难题
Synthetic Defect Image Generation for Power Line Insulator Inspection Using Multimodal Large Language Models
- 用多模态大模型根据图文提示生成缺陷图像
- 合成数据经筛选后使测试F1提升至0.739(原0.615)
- 适合无法获取更多真实缺陷数据的工业检测场景
电力公司越来越多地依赖无人机图像进行事件后和常规巡检,但训练准确的缺陷分类器仍面临挑战,因为缺陷样本稀少,且巡检数据集往往有限或专有。本文提出一种无需训练的图像生成方法,利用现成的多模态大语言模型(MLLM),基于视觉参考和文本提示生成缺陷图像。通过双参考条件增强多样性,结合轻量级人工验证与提示优化提升标签准确性,并采用基于嵌入距离的筛选规则,从真实训练集类别中心计算距离,筛选合成数据池。在公开陶瓷绝缘子缺陷分类数据集(壳体与釉面缺陷)上评估,真实训练集仅104张,验证集152张,测试集308张,低数据条件下,将10%真实数据与嵌入筛选后的合成数据混合,使测试F1得分从0.615提升至0.739(相对提升20%),相当于估计4–5倍的数据效率增益,且在更强骨干网络和冻结特征线性探测器上仍保持增益。结果表明,该方法为真实缺陷采集困难时提供了一条实用、低门槛的缺陷识别优化路径。
原文摘要 · Abstract (English)
Utility companies increasingly rely on drone imagery for post-event and routine inspection, but training accurate defect-type classifiers remains difficult because defect examples are rare and inspection datasets are often limited or proprietary. We address this data-scarcity setting by using an off-the-shelf multimodal large language model (MLLM) as a training-free image generator to synthesize defect images from visual references and text prompts. Our pipeline increases diversity via dual-reference conditioning, improves label fidelity with lightweight human verification and prompt refinement, and filters the resulting synthetic pool using an embedding-based selection rule based on distances to class centroids computed from the real training split. We evaluate on ceramic insulator defect-type classification (shell vs. glaze) using a public dataset with a realistic low training-data regime (104 real training images; 152 validation; 308 test). Augmenting the 10% real training set with embedding-selected synthetic images improves test F1 score (harmonic mean of precision and recall) from 0.615 to 0.739 (20% relative), corresponding to an estimated 4--5x data-efficiency gain, and the gains persist with stronger backbone models and frozen-feature linear-probe baselines. These results suggest a practical, low-barrier path for improving defect recognition when collecting additional real defects is slow or infeasible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。