用生成图像补全手套等防护装备,提升真实场景下手部检测精度。
Train, Test, Re-evaluate: Schedule-Sensitive Evaluation of Generative Data for Hand Detection

- 通过局部图像修复生成带手套等配件的合成手部数据
- 多阶段训练使检测[email protected]:0.95达最高,改善真实部署表现
- 适合需要高可靠性的工业安全场景手部检测应用
当目标图像稀缺、昂贵或存在偏差时,生成图像数据被越来越多用于补充或替代真实训练数据。在职业安全场景的手部检测中,公开数据集主要包含裸手图像,未能涵盖手套、纹身、饰品等个人防护装备带来的外观变化,导致部署时出现分布偏移。本文测试了仅对真实照片中的手部区域进行生成式修复以引入配件的方法,能否缩小这一差距并提升真实场景下的检测效果。在配对数据集上,我们对YOLOv8n手部检测器进行了六次实验(A-F),其中四次涉及训练(A, C, D, E),每组使用三个随机种子,评估模型在真实测试集和仅含戴手套的真实测试子集上的表现,并报告[email protected]与[email protected]:0.95结果及配对统计检验。两阶段实验:先在真实与合成数据上训练,再以较低学习率在纯真实数据上微调,相较于纯真实基线,在标准真实测试集上方向性提升了[email protected],同时缩小了戴手套情况下的分布外差距。三阶段实验在保持边界框紧致性方面最佳,是本研究中[email protected]:0.95最高的实验。合成数据在安全关键型手部检测中的有效性取决于训练流程,多阶段策略能从修复后的配件数据中提取显著的真实部署收益。
原文摘要 · Abstract (English)
Generated (or synthetic) image data is increasingly used to augment or replace real training datasets when target imagery is scarce, expensive, or biased. For hand detection, particularly in occupational safety settings, public datasets mostly contain bare hands. This under-represents the variation in hand appearance introduced by gloves, tattoos, jewelry, and other personal protective equipment, creating a distribution shift that safety-critical applications encounter at deployment. We test whether generative inpainting, editing only the hand region of a real photograph to introduce accessories, can close this shift gap and improve detection of real hands at deployment. On a paired dataset of real images and their synthetic counterparts, we evaluate YOLOv8n hand detectors across six experiments (A-F), four of which involve training (A, C, D, E) under three random seeds each, evaluate them on a real test set and on a real-gloves-only test split, and report the mean average precision (mAP) at two overlap thresholds ([email protected] and [email protected]:0.95) along with paired statistical tests. A two-stage experiment: train on real U synthetic data, then fine-tune the resulting weights on real-only at a lower learning rate, directionally improves [email protected] compared to the real-only baseline model on the standard real test set, and narrows the real-gloves out-of-distribution gap. Another three-stage experiment preserves box-tightness best, achieving the highest [email protected]:0.95 among experiments in the study. The synthetic-data utility for safety-critical hand detection depends on the training procedure, and simple multi-stage experiments extract substantial real-deployment benefit from inpainted accessory data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。