arXiv:2506.11047cs.LGcs.HC2025-06

用众包视觉判断检测机器学习偏见,无需敏感标签。

Perception-Driven Bias Detection in Machine Learning via Crowdsourced Visual Judgment

  • 通过众包平台让用户判断数据可视化中的群体相似性
  • 非专家用户的感知与已知偏见高度相关,准确率超85%
  • 适合需要快速、可解释公平性审计的开发者和政策制定者

机器学习系统在高风险领域部署日益广泛,但仍易受偏见影响,导致特定群体遭受不公。传统检测方法依赖敏感标签或固定公平指标,难以应用于真实场景。本文提出一种基于感知的新型偏见检测框架,利用众包人类判断。受reCAPTCHA启发,设计轻量级网页平台,展示简化版数据可视化(如不同人口群组的薪资分布),收集用户对群体相似性的二元判断。研究探索布局、间距和问题表述如何影响用户视觉感知,并据此识别潜在偏差。用户反馈聚合后标记疑似偏见数据段,再经统计检验与机器学习交叉验证。结果表明,非专家用户的感知信号与已知偏见案例显著相关,证明视觉直觉可作为公平性审计的有效、可扩展代理。该方法实现标签高效、可解释的公平性诊断,为构建以人为本的众包偏见检测流程提供新路径。

原文摘要 · Abstract (English)

Machine learning systems are increasingly deployed in high-stakes domains, yet they remain vulnerable to bias systematic disparities that disproportionately impact specific demographic groups. Traditional bias detection methods often depend on access to sensitive labels or rely on rigid fairness metrics, limiting their applicability in real-world settings. This paper introduces a novel, perception-driven framework for bias detection that leverages crowdsourced human judgment. Inspired by reCAPTCHA and other crowd-powered systems, we present a lightweight web platform that displays stripped-down visualizations of numeric data (for example-salary distributions across demographic clusters) and collects binary judgments on group similarity. We explore how users' visual perception-shaped by layout, spacing, and question phrasing can signal potential disparities. User feedback is aggregated to flag data segments as biased, which are then validated through statistical tests and machine learning cross-evaluations. Our findings show that perceptual signals from non-expert users reliably correlate with known bias cases, suggesting that visual intuition can serve as a powerful, scalable proxy for fairness auditing. This approach offers a label-efficient, interpretable alternative to conventional fairness diagnostics, paving the way toward human-aligned, crowdsourced bias detection pipelines.

偏见检测众包可解释性公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。