arXiv:2409.16089cs.CVcs.AI2024-09被引 8

让人脸识别模型能用自然语言解释决策过程,支持交互问答。

From Pixels to Words: Leveraging Explainability in Face Recognition through Interactive Natural Language Processing

  • 结合XAI热力图与BERT模型生成自然语言解释
  • 用户可交互提问,获得面部区域贡献度等具体信息
  • 保持识别精度不变,适合高敏感场景使用

人脸识别系统因深度学习取得显著进展,但其缺乏可解释性,引发问责、公平性和可靠性担忧。本文提出一种交互式框架,融合模型无关的可解释人工智能(XAI)与自然语言处理技术,通过聊天机器人准确回答用户各类问题。所生成的解释以自然语言文本和可视化形式呈现,例如说明不同面部区域对两张人脸相似度的贡献。该方法基于对人脸图像输出显著性热力图的自动分析及BERT问答模型实现,为用户提供全面理解人脸识别决策的界面。框架具备交互性,支持根据用户知识背景追问以获取更精确信息。更重要的是,与以往研究不同,本方案不降低人脸识别性能。实验验证了方法的有效性,表明其在提升人脸识别系统可解释性与用户友好性方面的潜力,尤其适用于对决策透明性要求高的敏感场景。

原文摘要 · Abstract (English)

Face Recognition (FR) has advanced significantly with the development of deep learning, achieving high accuracy in several applications. However, the lack of interpretability of these systems raises concerns about their accountability, fairness, and reliability. In the present study, we propose an interactive framework to enhance the explainability of FR models by combining model-agnostic Explainable Artificial Intelligence (XAI) and Natural Language Processing (NLP) techniques. The proposed framework is able to accurately answer various questions of the user through an interactive chatbot. In particular, the explanations generated by our proposed method are in the form of natural language text and visual representations, which for example can describe how different facial regions contribute to the similarity measure between two faces. This is achieved through the automatic analysis of the output's saliency heatmaps of the face images and a BERT question-answering model, providing users with an interface that facilitates a comprehensive understanding of the FR decisions. The proposed approach is interactive, allowing the users to ask questions to get more precise information based on the user's background knowledge. More importantly, in contrast to previous studies, our solution does not decrease the face recognition performance. We demonstrate the effectiveness of the method through different experiments, highlighting its potential to make FR systems more interpretable and user-friendly, especially in sensitive applications where decision-making transparency is crucial.

人脸识别可解释AI自然语言交互式

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。