提出可解释方法LEAM,识别面部识别中个体特异的敏感区域。
Explainable AI for Analyzing Person-Specific Patterns in Facial Recognition Tasks
- 通过激活映射定位个体面部关键识别区域,不攻击模型而是解析其决策机制。
- 同一人图像间激活模式相似度(0.32-0.57)远高于不同人(0.04-0.13),证实个体特异性模式存在。
- 仅需1%关键像素即可有效验证,结果跨模型通用,适合隐私保护系统设计。
面部识别系统的普及带来严重隐私风险,亟需有效应对措施。现有对抗方法多采用通用策略,未能适配个体面部特征,限制了效果与隐蔽性。本文提出层嵌入激活映射(LEAM)技术,从个体层面识别对识别贡献最大的面部区域。不同于旨在欺骗系统的对抗攻击,LEAM是一种可解释性方法,用于理解模型工作原理,为未来隐私保护研究提供依据。我们将LEAM与人脸解析器结合,分析1000名个体在9个预训练面部识别模型上的数据。结果显示,尽管不同模型层关注区域差异显著,但整体激活模式在各类架构中普遍聚焦相似面部区域;同一人图像间的激活模式相似度(巴塔查里亚系数:0.32–0.57)远高于不同人(0.04–0.13),验证了个体特异性识别模式的存在。分析表明,模型主要关注面部中央区域(鼻区占关键识别区域的18.9%–29.7%),同时分散关注多个面部片段。通过遮蔽验证确认,仅使用LEAM识别出的1%最相关像素,即可实现有效判断,且该选择在不同模型间具有迁移性。研究成果为基于个体特征定制的隐私保护系统奠定了基础。
原文摘要 · Abstract (English)
The proliferation of facial recognition systems presents major privacy risks, driving the need for effective countermeasures. Current adversarial techniques apply generalized methods rather than adapting to individual facial characteristics, limiting their effectiveness and inconspicuousness. In this work, we introduce Layer Embedding Activation Mapping (LEAM), a novel technique that identifies which facial areas contribute most to recognition at an individual level. Unlike adversarial attack methods that aim to fool recognition systems, LEAM is an explainability technique designed to understand how these systems work, providing insights that could inform future privacy protection research. We integrate LEAM with a face parser to analyze data from 1000 individuals across 9 pre-trained facial recognition models. Our analysis reveals that while different layers within facial recognition models vary significantly in their focus areas, these models generally prioritize similar facial regions across architectures when considering their overall activation patterns, which show significantly higher similarity between images of the same individual (Bhattacharyya Coefficient: 0.32-0.57) vs. different individuals (0.04-0.13), validating the existence of person-specific recognition patterns. Our results show that facial recognition models prioritize the central region of face images (with nose areas accounting for 18.9-29.7% of critical recognition regions), while still distributing attention across multiple facial fragments. Proper selection of relevant facial areas was confirmed using validation occlusions, based on just 1% of the most relevant, LEAM-identified, image pixels, which proved to be transferable across different models. Our findings establish the foundation for future individually tailored privacy protection systems centered around LEAM's choice of areas to be perturbed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。