首次系统评估公平排序对RAG的影响,发现公平性不牺牲性能还能提升引用平衡。
Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation
- 在12个RAG模型中引入公平排序机制,优化相关结果的均衡曝光。
- 公平排序使生成内容中的来源引用更均衡,且不降低检索与生成质量。
- 适合关注AI公平性、可解释性和责任生成的研究者与开发者。
尽管检索在检索增强生成(RAG)系统中至关重要,但现有研究大多忽视了公平排序领域,未考虑所有利益相关方的需求。本文首次系统评估集成公平感知排序的RAG系统,同时关注排名公平性与溯源公平性,确保生成内容中引用的来源得到公平曝光。评估聚焦于物品层面的公平性,即相关项在检索结果中的均衡暴露,并探究其对系统有效性及用户最终看到的生成输出中来源归属的影响。通过在七个不同任务上测试十二个RAG模型,我们发现引入公平排序通常维持甚至提升检索与生成质量,反驳了公平性会损害性能的普遍观点。此外,公平检索实践显著改善了最终回复中的来源引用平衡,确保生成器公正引用所依赖的源。研究强调了检索与生成中物品层面公平性的关键作用,为负责任且公平的RAG系统奠定基础,并指引未来公平排序与溯源研究方向。
原文摘要 · Abstract (English)
Despite the central role of retrieval in retrieval-augmented generation (RAG) systems, much of the existing research on RAG overlooks the well-established field of fair ranking and fails to account for the interests of all stakeholders involved. In this paper, we conduct the first systematic evaluation of RAG systems that integrate fairness-aware rankings, addressing both ranking fairness and attribution fairness, which ensures equitable exposure of the sources cited in the generated content. Our evaluation focuses on measuring item-side fairness, specifically the fair exposure of relevant items retrieved by RAG systems, and investigates how this fairness impacts both the effectiveness of the systems and the attribution of sources in the generated output that users ultimately see. By experimenting with twelve RAG models across seven distinct tasks, we show that incorporating fairness-aware retrieval often maintains or even enhances both ranking quality and generation quality, countering the common belief that fairness compromises system performance. Additionally, we demonstrate that fair retrieval practices lead to more balanced attribution in the final responses, ensuring that the generator fairly cites the sources it relies on. Our findings underscore the importance of item-side fairness in retrieval and generation, laying the foundation for responsible and equitable RAG systems and guiding future research in fair ranking and attribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。