用游戏引擎生成假图训练视觉模型,能大幅减少真实数据需求。
The effects of using created synthetic images in computer vision training
- 用UE渲染合成图像补充数据集,降低采集成本
- 合成图像占比超60%时,模型准确率差距缩至1-2%
- 仅需10%真实数据即可达到传统50-70%的效果
本文研究使用Unreal Engine 4(UE)等渲染引擎生成合成图像,在图像充足和稀缺场景下对深度计算机视觉(CV)模型训练的增益。合成图像可提供近乎无限、可复现、灵活且低成本的训练数据,避免网络污染风险。通过两个二分类任务(猫狗识别与焊缝缺陷检测),测试了在不同规模的模型(VGG16与MobileNetV3-small)上加入合成图像后的测试精度变化。结果表明:当合成图像占比超过60%时,测试-训练精度差缩小至约1-2%,但测试精度未显著优于纯真实数据;而向纯合成数据集中添加少于10%的真实图像,分类错误率减半,后续继续提升。在所测试场景中,合成图像使真实数据使用量从传统50-70%降至10%。论文还提出利用预训练模型作为审计器评估合成图像质量的方法,为数据稀缺项目提供合成图像生成与应用指南。
原文摘要 · Abstract (English)
This paper investigates how rendering engines, like Unreal Engine 4 (UE), can be used to create synthetic images to supplement datasets for deep computer vision (CV) models in image abundant and image limited use cases. Using rendered synthetic images from UE can provide developers and businesses with a method of accessing nearly unlimited, reproducible, agile, and cheap training sets for their customers and applications without the threat of poisoned images from the internet or the cost of collecting them. The validity of these generated images are examined by testing the change in model test accuracy in two different sized CV models across two binary classification cases (Cat vs Dog and Weld Defect Detection). In addition, this paper provides an implementation of how to measure the quality of synthetic images by using pre-trained CV models as auditors. Results imply that for large (VGG16) and small (MobileNetV3-small) parameter deep CV models, adding >60% additional synthetic images to a real image dataset during model training can narrow the test-training accuracy gap to ~1-2% without a conclusive effect on test accuracy compared to using real world images alone. Likewise, adding <10% additional real training images to synthetic only training sets decreased the classification error rate in half, then decreasing further when adding more real training images. For these cases tested, using synthetic images from rendering engines allow researchers to only use 10% of their real images during training, compared to the traditional 50-70%. This research serves as an example of how to create synthetic images, guidelines on how to use the images, potential restrictions and possible performance improvements for data-scarce projects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。