arXiv:2607.24898cs.CVcs.AI2026-07

现有图文生成毒性检测对边缘群体无效,需定制化社区安全标准。

Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed

论文配图:Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed
图 1 · 摘自论文原文
  • 为侏儒症和视障人群制定专属安全准则,替代通用检测。
  • 通用模型在新准则下零样本表现差(F1低于0.37),近乎随机。
  • 提示工程与轻量微调可提升检测效果,但仍远逊于通用模型。

当前文本到图像生成的毒性检测采用统一模型和固定安全规则,无法有效保护边缘群体。实证发现,约35%被标记为安全的图像,在残障群体眼中仍属有害。本文主张开展社区特定毒性检测(CTD)。我们与残障专家合作,为侏儒症和盲/低视力群体制定安全准则,并基于2400张标注的T2I生成图像数据集验证其可行性。结果表明,大模型和现有通用检测器在零样本下表现极差(F1分别为0.32和0.37),几乎等同随机猜测。而基于提示的适应方法(ICL、VQA)显著提升性能(GPT-4o:F1 0.50 和 0.78),参数高效微调在小模型(0.5b–7b)上也取得进步(最佳F1 0.48 和 0.59),仅需少于100个示例。然而,整体性能仍远低于通用检测的F1 ≈ 0.9,凸显该方向的挑战与研究紧迫性。

原文摘要 · Abstract (English)

State-of-the-art toxicity detectors for text-to-image generation adopt a one-size-fits-all approach: a single universal model applying fixed safety guidelines to all users. Our empirical evidence shows that these detectors fail to shield marginalized communities: approximately 35% of generated images labeled safe are considered harmful by disability communities. In this position paper, we argue for community-specific toxicity detection (CTD). To demonstrate its feasibility, we collaborate with disability experts to develop safety guidelines for two communities: dwarfism and blind/low vision. Using a dataset of 2,400 annotated T2I-generated images we demonstrate that both large vision-language models and existing general-purpose toxicity detectors catastrophically fail to recognize harmful content under these guidelines in zero-shot settings with F1 score lower than random guessing (F1 0.32 and 0.37). Promisingly, prompt-based adaptation methods (ICL, VQA) substantially improve harm detection performance (GPT-4o: F1 0.50 and 0.78), while parameter-efficient fine-tuning improves smaller models (0.5b-7b with best F1 0.48 and 0.59) with less than 100 demonstrations, but remains sensitive to evolving guidelines. Despite these gains, CTD performance remains far below F1 $\approx 0.9$ achieved for general-purpose toxicity detection, highlighting the challenge and the need for sustained research effort.

毒性检测社区适配残障友好

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。