提出新方法ALBAR,同时消除动作识别中的背景与前景偏差。
ALBAR: Adversarial Learning approach to mitigate Biases in Action Recognition
- 用对抗性损失让静态片段分类概率均匀化,削弱背景依赖。
- 在HMDB51上综合去偏性能提升超12个百分点,达新最优。
- 发现UCF101评估协议存在背景泄露问题,提出更精细的演员分割边界。
机器学习模型中的偏差可能导致不公平决策,尽管图像与文本领域已有深入研究,动作识别中的偏差仍被低估。现有模型常受背景偏差(基于背景线索推断动作)和前景偏差(依赖主体外观)影响,对自动驾驶或辅助生活监控等真实应用构成威胁。以往方法多针对背景偏差使用特定增强,而本文全面研究前景与背景偏差。提出ALBAR,一种无需先验偏差知识的对抗训练方法:对采样静态片段(所有帧相同)施加对抗交叉熵损失,并通过熵最大化损失使类别概率趋于均匀;同时引入梯度惩罚损失以正则化去偏过程。在主流背景与前景偏差评估协议上验证,ALBAR达到新最优,且在HMDB51上综合去偏性能提升超过12%。此外,我们指出当前UCF101评估协议存在背景泄露问题,提供预测捷径,无法准确衡量模型去偏能力。为此,我们提出更细粒度的演员分割边界,改进后方法仍优于现有方案。
原文摘要 · Abstract (English)
Bias in machine learning models can lead to unfair decision making, and while it has been well-studied in the image and text domains, it remains underexplored in action recognition. Action recognition models often suffer from background bias (i.e., inferring actions based on background cues) and foreground bias (i.e., relying on subject appearance), which can be detrimental to real-life applications such as autonomous vehicles or assisted living monitoring. While prior approaches have mainly focused on mitigating background bias using specialized augmentations, we thoroughly study both foreground and background bias. We propose ALBAR, a novel adversarial training method that mitigates foreground and background biases without requiring specialized knowledge of the bias attributes. Our framework applies an adversarial cross-entropy loss to the sampled static clip (where all the frames are the same) and aims to make its class probabilities uniform using a proposed entropy maximization loss. Additionally, we introduce a gradient penalty loss for regularization against the debiasing process. We evaluate our method on established background and foreground bias protocols, setting a new state-of-the-art and strongly improving combined debiasing performance by over 12% absolute on HMDB51. Furthermore, we identify an issue of background leakage in the existing UCF101 protocol for bias evaluation which provides a shortcut to predict actions and does not provide an accurate measure of the debiasing capability of a model. We address this issue by proposing more fine-grained segmentation boundaries for the actor, where our method also outperforms existing approaches. Project Page: https://joefioresi718.github.io/ALBAR_webpage/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。