不同数据和算法配置下,推荐系统偏见结论截然不同,需明确说明条件。
On the challenges of studying bias in Recommender Systems: A UserKNN case study
- 用合成数据与UserKNN框架结合,测试偏见传播
- 五组数据与五种配置组合,结果差异显著
- 提醒研究者报告偏见时必须说明数据与参数
关于推荐系统传播偏见的论述往往难以验证或证伪。现有研究多依赖少量公开数据集,且算法实现细节常未明确说明,可能影响偏见评估。本文以UserKNN为例,探讨测量与报告流行度偏见的挑战。基于UserKNN的功能特性,我们识别出可能影响偏见的数据特征,并生成五组合成数据。同时,梳理文献中UserKNN的多种配置差异。在五组数据与五种配置组合下评估流行度偏见,发现数据特性与算法配置的联合效应会导致对偏见传播的不同结论。这表明,在报告和解读推荐系统的偏见时,必须明示算法配置与数据属性。
原文摘要 · Abstract (English)
Statements on the propagation of bias by recommender systems are often hard to verify or falsify. Research on bias tends to draw from a small pool of publicly available datasets and is therefore bound by their specific properties. Additionally, implementation choices are often not explicitly described or motivated in research, while they may have an effect on bias propagation. In this paper, we explore the challenges of measuring and reporting popularity bias. We showcase the impact of data properties and algorithm configurations on popularity bias by combining synthetic data with well known recommender systems frameworks that implement UserKNN. First, we identify data characteristics that might impact popularity bias, based on the functionality of UserKNN. Accordingly, we generate various datasets that combine these characteristics. Second, we locate UserKNN configurations that vary across implementations in literature. We evaluate popularity bias for five synthetic datasets and five UserKNN configurations, and offer insights on their joint effect. We find that, depending on the data characteristics, various UserKNN configurations can lead to different conclusions regarding the propagation of popularity bias. These results motivate the need for explicitly addressing algorithmic configuration and data properties when reporting and interpreting bias in recommender systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。