用机器学习识别争议内容,自动触发多人复审,提升社区审核一致性。
Venire: A Machine Learning-Guided Panel Review System for Community Content Moderation
- 基于历史数据训练模型,筛选高争议内容自动进入多人评审流程。
- 实验显示系统显著提升审核一致性,发现此前未察觉的分歧。
- 适合希望改善人工审核公平性与透明度的平台或团队使用。
社区内容审核常假设审核团队具有一致判断,但研究发现审核员之间存在可观测的分歧。问题不在于主观差异本身——多样性本有优势;而在于资源限制导致多数决策由单人完成,造成结果不一致,令社区成员感到困扰。为此,本文提出 Venire,一个基于机器学习的面板复审系统,用于 Reddit 内容审核。该系统利用日志数据训练模型,识别出最可能引发分歧的案例,并将其快速推送至多人复审流程。理想情况下,Venire 可帮助团队发现并解决原本被忽视的争议。我们通过三项研究验证系统:对审核员的形成性访谈、在两个数据集上的技术评估,以及真实案例的思考过程研究。定量结果显示,系统有效提升决策一致性并揭示潜在分歧;定性分析表明,审核员在处理复杂案例时更具信心。Venire 开创了人机协同审核新范式,从替代人类转向增强人类判断。
原文摘要 · Abstract (English)
Research into community content moderation often assumes that moderation teams govern with a single, unified voice. However, recent work has found that moderators disagree with one another at modest, but concerning rates. The problem is not the root disagreements themselves. Subjectivity in moderation is unavoidable, and there are clear benefits to including diverse perspectives within a moderation team. Instead, the crux of the issue is that, due to resource constraints, moderation decisions end up being made by individual decision-makers. The result is decision-making that is inconsistent, which is frustrating for community members. To address this, we develop Venire, an ML-backed system for panel review on Reddit. Venire uses a machine learning model trained on log data to identify the cases where moderators are most likely to disagree. Venire fast-tracks these cases for multi-person review. Ideally, Venire allows moderators to surface and resolve disagreements that would have otherwise gone unnoticed. We conduct three studies through which we design and evaluate Venire: a set of formative interviews with moderators, technical evaluations on two datasets, and a think-aloud study in which moderators used Venire to make decisions on real moderation cases. Quantitatively, we demonstrate that Venire is able to improve decision consistency and surface latent disagreements. Qualitatively, we find that Venire helps moderators resolve difficult moderation cases more confidently. Venire represents a novel paradigm for human-AI content moderation, and shifts the conversation from replacing human decision-making to supporting it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。