同时处理图文偏见,让政治内容更中立。
Multimodal Political Bias Identification and Neutralization
- 用CLIP对齐图文语义,用ViT评估图像偏见。
- 通过BERT识别并中和文本中的偏见词汇。
- 可减少图文偏见,适合内容审核与公平性研究。
由于政治信息回音室的存在,必须从政治文章的文本和图像中检测并消除主观偏见和情绪化语言。然而,以往工作仅关注文本偏见,忽视了图像同样具有强大的信息传达能力。为此,我们提出一个融合文本与图像偏见的四步模型:图像-文本对齐利用CLIP实现语义对齐;图像偏见评分通过ViT分类器确定;文本去偏则通过BERT识别并中和偏见词句;最终步骤结合三者结果,以偏见得分对比替换原始图文为中性版本。实验表明,该方法前景良好,文本去偏策略能有效识别大量潜在偏见词句,ViT模型训练表现优异,语义对齐模型也具高效性。但需更多训练时间和资源以提升效果。还设计了人工评估环节,确保生成图文的语义一致性。
原文摘要 · Abstract (English)
Due to the presence of political echo chambers, it becomes imperative to detect and remove subjective bias and emotionally charged language from both the text and images of political articles. However, prior work has focused on solely the text portion of the bias rather than both the text and image portions. This is a problem because the images are just as powerful of a medium to communicate information as text is. To that end, we present a model that leverages both text and image bias which consists of four different steps. Image Text Alignment focuses on semantically aligning images based on their bias through CLIP models. Image Bias Scoring determines the appropriate bias score of images via a ViT classifier. Text De-Biasing focuses on detecting biased words and phrases and neutralizing them through BERT models. These three steps all culminate to the final step of debiasing, which replaces the text and the image with neutralized or reduced counterparts, which for images is done by comparing the bias scores. The results so far indicate that this approach is promising, with the text debiasing strategy being able to identify many potential biased words and phrases, and the ViT model showcasing effective training. The semantic alignment model also is efficient. However, more time, particularly in training, and resources are needed to obtain better results. A human evaluation portion was also proposed to ensure semantic consistency of the newly generated text and images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。