构建多模态新闻偏见检测框架,让模型学会看图识偏。
ViLBias: Detecting and Reasoning about Bias in Multimodal Content
- 设计图文问答式评测基准,支持文本与图像联合推理
- 图文结合使检测准确率提升3%-5%,大模型更擅长发现隐性偏见
- 参数高效微调仅用不到5%参数即可接近全量训练效果
检测多模态新闻中的偏见需要能够对图文对进行推理的模型,而不仅仅是分类文本。为此,我们提出ViLBias,一个基于VQA风格的基准数据集与框架,用于检测和推理多模态新闻中的偏见。该数据集包含来自不同媒体的40,945个图文对,每个样本均通过两阶段LLM标注流程(层级多数投票+人工验证)标注偏见标签及简洁理由。我们在封闭式分类与开放式推理(oVQA)任务上评估了小型语言模型(SLMs)、大型语言模型(LLMs)和视觉-语言模型(VLMs),并比较了参数高效微调策略。结果显示,结合图像可使检测准确率提升3–5%,且LLMs/VLMs比SLMs更能捕捉细微的表述偏差与图文不一致。参数高效方法(LoRA/QLoRA/Adapters)在仅使用<5%可训练参数的情况下,恢复了97–99%的全量微调性能。在oVQA任务中,推理准确率为52–79%,忠实度为68–89%,两者均通过指令微调得到改善;封闭式准确率与推理能力高度相关(r = 0.91)。ViLBias提供了一个可扩展的基准和强大的基线,用于多模态偏见检测与理由质量评估。
原文摘要 · Abstract (English)
Detecting bias in multimodal news requires models that reason over text--image pairs, not just classify text. In response, we present ViLBias, a VQA-style benchmark and framework for detecting and reasoning about bias in multimodal news. The dataset comprises 40,945 text--image pairs from diverse outlets, each annotated with a bias label and concise rationale using a two-stage LLM-as-annotator pipeline with hierarchical majority voting and human-in-the-loop validation. We evaluate Small Language Models (SLMs), Large Language Models (LLMs), and Vision--Language Models (VLMs) across closed-ended classification and open-ended reasoning (oVQA), and compare parameter-efficient tuning strategies. Results show that incorporating images alongside text improves detection accuracy by 3--5\%, and that LLMs/VLMs better capture subtle framing and text--image inconsistencies than SLMs. Parameter-efficient methods (LoRA/QLoRA/Adapters) recover 97--99\% of full fine-tuning performance with $<5\%$ trainable parameters. For oVQA, reasoning accuracy spans 52--79\% and faithfulness 68--89\%, both improved by instruction tuning; closed accuracy correlates strongly with reasoning ($r = 0.91$). ViLBias offers a scalable benchmark and strong baselines for multimodal bias detection and rationale quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。