arXiv:2607.14423cs.CVeess.IV2026-07

冻结的DINOv3模型能无训练识别面部区域,适合做零样本人脸分析。

Emergent Region-Level Facial Correspondence in Frozen Vision Foundation Models

论文配图:Emergent Region-Level Facial Correspondence in Frozen Vision Foundation Models
图 1 · 摘自论文原文
  • 用DINOv3特征构建跨人面部区域坐标系,无需微调
  • 跨身份匹配准确率达83.0%,时间追踪达95.5%
  • 中间层特征最适合细粒度人脸分析,优于末层

冻结的自监督视觉模型可对通用物体进行局部对齐,但其是否适用于人脸尚不明确——人脸共享全局结构,但个体外观差异显著。本文测试冻结的DINOv3特征能否建立区域级面部坐标系统:即在不进行人脸特定训练的情况下,眼睛、眉毛、鼻子、嘴巴、皮肤和头发等区域在不同人之间及随时间变化时仍可区分。基于DINOv3 ViT-L/16的补丁嵌入,仅使用FaRL作为面部区域标签接口,在200段CelebDF-v2真实视频上评估了跨身份最近邻匹配与时间标签传播。DINOv3在非约束条件下实现83.0%的区域级语义准确率,远高于23.0%的面积加权随机基线;无学习时间模块下时间追踪准确率达95.5%。无FaRL控制组下降至0.9%,表明FaRL提供语义初始化,而DINOv3提供密集空间对应。最强对应出现在中间层:第18层相同区域与跨区域判别比为4.93:1,末层仅为1.48:1。相比CLIP ViT-L/14,DINOv3总体优势较小,但在解剖区域上高出16.8个百分点,说明图像级对比学习捕捉粗略布局,却无法处理精细解剖身份。结果确立冻结的DINOv3是区域级面部对应的强大零样本表示,并指出中间自监督特征层最适于密集人脸分析。

原文摘要 · Abstract (English)

Frozen self-supervised vision models can align parts of generic objects, but it remains unclear whether this correspondence extends to human faces, where global layout is shared while identity-specific appearance varies sharply. We test whether frozen DINOv3 features define a region-level facial coordinate system: a feature space in which eyes, brows, nose, mouth, skin, and hair remain distinguishable across people and across time without face-specific training. Using DINOv3 ViT-L/16 patch embeddings and FaRL only as a face-part labeling interface, we evaluate cross-identity nearest-neighbor matching and temporal label propagation on 200 CelebDF-v2 real videos. DINOv3 achieves 83.0% region-level semantic accuracy under unconstrained cross-identity matching, compared with a 23.0% area-weighted random baseline, and 95.5% temporal tracking accuracy without a learned temporal module. A no-FaRL control collapses to 0.9%, showing that FaRL supplies semantic initialization while DINOv3 supplies dense spatial correspondence. The strongest correspondence appears at an intermediate layer: block 18 gives a 4.93x same-region versus cross-region discrimination ratio, compared with 1.48x at the final block. Against CLIP ViT-L/14, DINOv3 shows only a small aggregate advantage but a +16.8 pp gain on anatomical regions, indicating that image-level contrastive supervision captures coarse facial layout but not fine-grained anatomical identity. These results establish frozen DINOv3 as a strong zero-shot representation for region-level facial correspondence and identify intermediate self-supervised features as the most useful layer for dense face analysis.

面部对应DINOv3零样本自监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。