提出用帕累托前沿距离评估推荐系统公平性与相关性的联合表现
Joint Evaluation of Fairness and Relevance in Recommender Systems with Pareto Frontier
- 基于帕累托前沿计算公平性与相关性可同时达到的最优边界
- 实验显示现有指标与帕累托最优解不一致，新方法更稳定可靠
- 适合关注公平性与效果平衡的研究者和工程实践者
公平性与相关性是推荐系统（RS）的两个关键维度。传统评估方法要么分别使用独立指标，导致最优模型不一致；要么采用单一综合指标，但这些指标常为临时设计，与传统相关性指标（如NDCG）关联性差。为此，本文提出一种新的联合评估方法——帕累托前沿距离（DPFR）：在给定用户-物品交互数据下，计算现有相关性与公平性指标的帕累托前沿，并以模型性能到前沿的距离作为联合表现度量。该方法模块化且直观，可兼容现有指标。在4种推荐模型、3种重排序策略和6个数据集上的实验表明，现有指标与帕累托最优解存在不一致，而DPFR展现出更强的鲁棒性和理论基础，是评估公平性与相关性更可靠的联合指标。代码已开源。
原文摘要 · Abstract (English)
Fairness and relevance are two important aspects of recommender systems (RSs). Typically, they are evaluated either (i) separately by individual measures of fairness and relevance, or (ii) jointly using a single measure that accounts for fairness with respect to relevance. However, approach (i) often does not provide a reliable joint estimate of the goodness of the models, as it has two different best models: one for fairness and another for relevance. Approach (ii) is also problematic because these measures tend to be ad-hoc and do not relate well to traditional relevance measures, like NDCG. Motivated by this, we present a new approach for jointly evaluating fairness and relevance in RSs: Distance to Pareto Frontier (DPFR). Given some user-item interaction data, we compute their Pareto frontier for a pair of existing relevance and fairness measures, and then use the distance from the frontier as a measure of the jointly achievable fairness and relevance. Our approach is modular and intuitive as it can be computed with existing measures. Experiments with 4 RS models, 3 re-ranking strategies, and 6 datasets show that existing metrics have inconsistent associations with our Pareto-optimal solution, making DPFR a more robust and theoretically well-founded joint measure for assessing fairness and relevance. Our code: https://github.com/theresiavr/DPFR-recsys-evaluation
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。