arXiv:2602.00635cs.CVcs.AI2026-02

无需标注遮挡掩码,用自监督提示学习实现精准人脸遮挡分割。

S$^3$POT: Contrast-Driven Face Occlusion Segmentation via Self-Supervised Prompt Learning

  • 利用生成模型重构面部并生成参考图,构建对比线索。
  • 通过跨注意力增强特征,自动生成正负提示筛选最优掩码。
  • 完全无需遮挡真实标签,适合缺乏标注数据的场景。

现有面部解析方法常将遮挡误判为面部部件,因遮挡是高层概念而非具体物体类别,难以收集覆盖所有遮挡类别的真实数据,且标注掩码成本极高。为此,本文提出S$^3$POT,一种结合面部生成与自监督空间提示的对比驱动框架,实现遮挡分割。该框架基于两大洞察:1)现代面部生成器能真实重建被遮挡区域,在保留面部结构的同时消除遮挡;2)基础分割模型(如SAM)在获得合适提示时可生成精确掩码。S$^3$POT包含三个模块:参考图像生成(RF)、特征增强(FE)和提示选择(PS)。首先,RF利用解析掩码的结构引导生成参考图像;其次,FE通过对比原始图与参考图的特征令牌,生成初始提示,并通过交叉注意力修改图像特征;最后,基于增强特征,PS构建正负提示集,经自注意力网络筛选后输入掩码解码器。整个网络在三个新颖且互补的目标函数指导下训练,无需遮挡真实掩码。在专门收集的数据集上进行的大量实验表明,S$^3$POT性能优越,各模块有效。

原文摘要 · Abstract (English)

Existing face parsing methods usually misclassify occlusions as facial components. This is because occlusion is a high-level concept, it does not refer to a concrete category of object. Thus, constructing a real-world face dataset covering all categories of occlusion object is almost impossible and accurate mask annotation is labor-intensive. To deal with the problems, we present S$^3$POT, a contrast-driven framework synergizing face generation with self-supervised spatial prompting, to achieve occlusion segmentation. The framework is inspired by the insights: 1) Modern face generators' ability to realistically reconstruct occluded regions, creating an image that preserve facial geometry while eliminating occlusion, and 2) Foundation segmentation models' (e.g., SAM) capacity to extract precise mask when provided with appropriate prompts. In particular, S$^3$POT consists of three modules: Reference Generation (RF), Feature enhancement (FE), and Prompt Selection (PS). First, a reference image is produced by RF using structural guidance from parsed mask. Second, FE performs contrast of tokens between raw and reference images to obtain an initial prompt, then modifies image features with the prompt by cross-attention. Third, based on the enhanced features, PS constructs a set of positive and negative prompts and screens them with a self-attention network for a mask decoder. The network is learned under the guidance of three novel and complementary objective functions without occlusion ground truth mask involved. Extensive experiments on a dedicatedly collected dataset demonstrate S$^3$POT's superior performance and the effectiveness of each module.

遮挡分割自监督提示学习面部生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。