用合成干预揭示表情识别模型依赖脸对称性的因果机制
Facing Asymmetry -- Uncovering the Causal Link between Facial Symmetry and Expression Classifiers using Synthetic Interventions
- 构建因果模型并设计合成干预框架，隔离对称性影响
- 17个表情分类器在对称性降低时输出激活均下降
- 为黑箱模型行为分析提供可解释的因果研究范式
理解面部表情对解读人类行为至关重要。当前端到端训练的黑箱模型表现优异，但其在分布外数据上的行为尚不明确，尤其在单侧面瘫患者中性能下降。我们假设面部对称性是影响模型决策的关键因素。本文基于因果推理构建结构因果模型，并提出一种合成干预框架，可在固定其他因素的前提下分析对称性对网络输出的影响。实验发现，所有17个表情分类器在对称性降低时输出激活显著下降，这一结果与健康人群及面瘫患者的真实数据表现一致。本研究为揭示黑箱模型行为背后的因果因素提供了典型案例。
原文摘要 · Abstract (English)
Understanding expressions is vital for deciphering human behavior, and nowadays, end-to-end trained black box models achieve high performance. Due to the black-box nature of these models, it is unclear how they behave when applied out-of-distribution. Specifically, these models show decreased performance for unilateral facial palsy patients. We hypothesize that one crucial factor guiding the internal decision rules is facial symmetry. In this work, we use insights from causal reasoning to investigate the hypothesis. After deriving a structural causal model, we develop a synthetic interventional framework. This approach allows us to analyze how facial symmetry impacts a network's output behavior while keeping other factors fixed. All 17 investigated expression classifiers significantly lower their output activations for reduced symmetry. This result is congruent with observed behavior on real-world data from healthy subjects and facial palsy patients. As such, our investigation serves as a case study for identifying causal factors that influence the behavior of black-box models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。