重构因果链,解决场景图生成中的虚假关联问题。
A Reverse Causal Framework to Mitigate Spurious Correlations for Debiasing Scene Graph Generation
- 将检测器与分类器的因果关系反转,以消除预测偏差。
- 在多个基准上实现最优平均召回率,优于现有方法。
- 适合关注图像理解中公平性与可解释性的研究者。
现有的两阶段场景图生成(SGG)框架通常先通过检测器提取关系特征,再由分类器进行分类,形成从检测到分类的因果链结构。然而,这种结构会导致检测输入与最终预测之间产生虚假相关性,即某一关系的预测受其他关系影响。这会引发至少两种可观察的偏差:尾部关系被误判为头部关系,前景关系被误判为背景关系;其中后者在文献中极少被讨论。为解决此问题,我们提出将因果链重构为反向因果结构,将分类器输入视为混杂因子,检测器输入和最终预测均作为因果变量。具体地,提出主动反向估计(ARE)来干预混杂因子,估计从最终预测到分类器输入的反向因果关系;进一步引入最大信息采样(MIS),利用关系信息增强反向因果估计。理论上,该方法可缓解SGG框架中的虚假相关性,从而消除由此产生的偏差。在多个主流基准及不同SGG框架上的综合实验表明,RcSGG达到了最先进的平均召回率。
原文摘要 · Abstract (English)
Existing two-stage Scene Graph Generation (SGG) frameworks typically incorporate a detector to extract relationship features and a classifier to categorize these relationships; therefore, the training paradigm follows a causal chain structure, where the detector's inputs determine the classifier's inputs, which in turn influence the final predictions. However, such a causal chain structure can yield spurious correlations between the detector's inputs and the final predictions, i.e., the prediction of a certain relationship may be influenced by other relationships. This influence can induce at least two observable biases: tail relationships are predicted as head ones, and foreground relationships are predicted as background ones; notably, the latter bias is seldom discussed in the literature. To address this issue, we propose reconstructing the causal chain structure into a reverse causal structure, wherein the classifier's inputs are treated as the confounder, and both the detector's inputs and the final predictions are viewed as causal variables. Specifically, we term the reconstructed causal paradigm as the Reverse causal Framework for SGG (RcSGG). RcSGG initially employs the proposed Active Reverse Estimation (ARE) to intervene on the confounder to estimate the reverse causality, \ie the causality from final predictions to the classifier's inputs. Then, the Maximum Information Sampling (MIS) is suggested to enhance the reverse causality estimation further by considering the relationship information. Theoretically, RcSGG can mitigate the spurious correlations inherent in the SGG framework, subsequently eliminating the induced biases. Comprehensive experiments on popular benchmarks and diverse SGG frameworks show the state-of-the-art mean recall rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。