提出群体公平性概念,确保学术会议中各研究社区不因被边缘化而受损。
Group Fairness in Peer Review
- 引入'核心'公平性概念,防止任何研究群体单方面退出大会获利。
- 证明在简单评审模型下总存在满足核心条件的分配方案。
- 基于CVPR和ICLR数据验证算法效果,优于现有分配方法。
大型会议如NeurIPS和AAAI汇聚了众多AI领域,但也导致部分研究社区的论文被分配给非本领域评审人,评审体验下降。尽管有人建议拆分大会为小会议,但这可能造成社区孤立,阻碍跨学科研究。本文提出一种名为‘核心’的群体公平性概念,要求每个可能的研究群体(研究人员子集)均不能通过单方面退出大会而获益。在简化评审模型下,我们证明了核心分配始终存在,并设计了一种高效算法来寻找此类分配。利用CVPR和ICLR的真实数据,我们将该算法与现有分配方法在多个指标上进行对比,验证了其优越性。
原文摘要 · Abstract (English)
Large conferences such as NeurIPS and AAAI serve as crossroads of various AI fields, since they attract submissions from a vast number of communities. However, in some cases, this has resulted in a poor reviewing experience for some communities, whose submissions get assigned to less qualified reviewers outside of their communities. An often-advocated solution is to break up any such large conference into smaller conferences, but this can lead to isolation of communities and harm interdisciplinary research. We tackle this challenge by introducing a notion of group fairness, called the core, which requires that every possible community (subset of researchers) to be treated in a way that prevents them from unilaterally benefiting by withdrawing from a large conference. We study a simple peer review model, prove that it always admits a reviewing assignment in the core, and design an efficient algorithm to find one such assignment. We use real data from CVPR and ICLR conferences to compare our algorithm to existing reviewing assignment algorithms on a number of metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。