用频率提示法提升大模型观点摘要的公平性
REFER: Mitigating Bias in Opinion Summarisation via Frequency Framed Prompting
- 将抽象概率转为具体频数，降低模型偏见
- 大模型+强推理指令下公平性提升显著
- 适合关注生成公平性的NLP研究者
人们表达多元观点，公正的摘要应全面呈现不同立场。以往基于大语言模型（LLMs）的观点摘要公平性研究依赖超参数调优或在提示中提供真实分布信息，但存在实际局限：用户通常无法修改默认参数，且准确分布信息常不可得。受认知科学启发——频数表示能减少人类统计推理中的系统性偏见，因其显式揭示参考群体并降低认知负荷——本研究探索频率提示法（REFER）是否可同样提升LLM观点摘要的公平性。通过系统实验对比多种提示框架，我们将提升人类推理效率的技术迁移到语言模型，使其相比抽象概率表示更能有效处理信息。结果表明，REFER能显著增强模型在观点摘要中的公平性，尤其在更大模型和更强推理指令下效果更明显。
原文摘要 · Abstract (English)
Individuals express diverse opinions, a fair summary should represent these viewpoints comprehensively. Previous research on fairness in opinion summarisation using large language models (LLMs) relied on hyperparameter tuning or providing ground truth distributional information in prompts. However, these methods face practical limitations: end-users rarely modify default model parameters, and accurate distributional information is often unavailable. Building upon cognitive science research demonstrating that frequency-based representations reduce systematic biases in human statistical reasoning by making reference classes explicit and reducing cognitive load, this study investigates whether frequency framed prompting (REFER) can similarly enhance fairness in LLM opinion summarisation. Through systematic experimentation with different prompting frameworks, we adapted techniques known to improve human reasoning to elicit more effective information processing in language models compared to abstract probabilistic representations.Our results demonstrate that REFER enhances fairness in language models when summarising opinions. This effect is particularly pronounced in larger language models and using stronger reasoning instructions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。