arXiv:2605.16113cs.CLcs.AI2026-05被引 1

无需微调,通过检索增强生成实现动态公平生成。

DebiasRAG: A Tuning-Free Path to Fair Generation in Large Language Models through Retrieval-Augmented Generation

论文配图:DebiasRAG: A Tuning-Free Path to Fair Generation in Large Language Models through Retrieval-Augmented Generation
图 1 · 摘自论文原文
  • 利用查询特定的偏见上下文反向生成公平约束。
  • 在不损失模型能力前提下,显著降低种族性别等社会偏见。
  • 适合需要实时公平性保障的应用场景,如客服与内容生成。

大语言模型(LLMs)虽具备卓越生成能力,但其训练数据中的知识可能导致幻觉、刻板印象及社会偏见,尤其在涉及种族、性别、年龄等维度时更为明显。以往方法依赖微调或提示工程,需额外资源且可能削弱原生能力,还难以适应动态偏见场景。本文提出DebiasRAG——一种无需微调、基于检索增强生成(RAG)的动态查询特定去偏框架。该框架包含三个阶段:(1)通过常规检索生成与查询相关的偏见候选上下文;(2)构建上下文候选池;(3)基于梯度更新对去偏引导上下文进行重排序。具体而言,系统预先离线准备偏见上下文,再结合查询逆向生成去偏上下文,作为输出的公平性约束。同时,从标准RAG文档库(如分块维基百科)中检索相关上下文。该方法在保留模型表达能力的同时,有效提升生成内容的公平性。

原文摘要 · Abstract (English)

Large language models (LLMs) have achieved unprecedented success due to their exceptional generative capabilities. However, because they depend on knowledge encapsulated from training corpora, they may produce hallucinations, stereotypes, and socially biased content. In particular, LLMs are prone to prejudiced responses involving race, gender, and age, which are collectively referred to as social biases. Prior studies have used fine-tuning and prompt engineering to mitigate such biases in LLMs, but these methods require additional training resources or domain knowledge to design the framework. Moreover, they may degrade the original capabilities of LLMs and often overlook the need for dynamic debiasing contexts for fairer inference. In this paper, we propose DebiasRAG, a novel tuning-free and dynamic query-specific debiasing framework based on retrieval-augmented generation (RAG). DebiasRAG improves fairness while preserving the intrinsic properties of LLMs, such as representation ability. DebiasRAG consists of three stages: (1) query-specific debiasing candidate generation; (2) context candidate pool construction; and (3) gradient-updated debiasing-guided context piece reranking. First, DebiasRAG leverages self-diagnosed bias contexts relevant to the query through regular retrieval, where the bias contexts are prepared offline by the DebiasRAG provider. Given the query-specific bias contexts, DebiasRAG reversely produces debiasing contexts, which are provided as additional fairness constraints for LLM outputs. Second, a regular RAG retrieval process produces query-related contexts from the regular RAG document database, such as a chunked Wikipedia dataset.

去偏RAG公平生成大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。