通过深度感知与多模态匹配,生成更真实的人脸检测训练数据。
Depth-Copy-Paste: Multimodal and Depth-Aware Compositing for Robust Face Detection
- 结合BLIP、CLIP与SAM3,实现语义一致且细节完整的前景提取。
- 利用深度图引导贴图位置,确保场景几何连续性与尺度匹配。
- 在复杂遮挡和光照下显著提升人脸检测性能,适合实际应用。
数据增强对提升人脸检测系统在遮挡、光照变化及复杂环境下的鲁棒性至关重要。传统复制粘贴增强常因前景提取不准确、场景几何不一致和背景语义不匹配而生成不真实样本。为此,我们提出深度感知的多模态复制粘贴框架(Depth Copy Paste),通过复制完整人体实例并粘贴到语义匹配的背景中,生成多样化且物理一致的训练样本。该方法首先利用BLIP与CLIP联合评估语义与视觉一致性,自动检索最适配的背景图像;为获取高质量前景掩码并保留面部细节,集成SAM3进行精确分割,并使用Depth-Anything仅提取非遮挡可见区域,避免污染面部纹理;为保证几何真实性,引入基于深度图的滑动窗口放置机制,在背景深度图上搜索具有最优深度连续性和尺度对齐的粘贴位置。生成的合成图像展现出自然的深度关系与更高视觉合理性。大量实验表明,相比传统复制粘贴与无深度增强方法,Depth Copy Paste 提供了更丰富真实的训练数据,显著提升了下游人脸检测任务的性能。
原文摘要 · Abstract (English)
Data augmentation is crucial for improving the robustness of face detection systems, especially under challenging conditions such as occlusion, illumination variation, and complex environments. Traditional copy paste augmentation often produces unrealistic composites due to inaccurate foreground extraction, inconsistent scene geometry, and mismatched background semantics. To address these limitations, we propose Depth Copy Paste, a multimodal and depth aware augmentation framework that generates diverse and physically consistent face detection training samples by copying full body person instances and pasting them into semantically compatible scenes. Our approach first employs BLIP and CLIP to jointly assess semantic and visual coherence, enabling automatic retrieval of the most suitable background images for the given foreground person. To ensure high quality foreground masks that preserve facial details, we integrate SAM3 for precise segmentation and Depth-Anything to extract only the non occluded visible person regions, preventing corrupted facial textures from being used in augmentation. For geometric realism, we introduce a depth guided sliding window placement mechanism that searches over the background depth map to identify paste locations with optimal depth continuity and scale alignment. The resulting composites exhibit natural depth relationships and improved visual plausibility. Extensive experiments show that Depth Copy Paste provides more diverse and realistic training data, leading to significant performance improvements in downstream face detection tasks compared with traditional copy paste and depth free augmentation methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。