不同提问方式导致AI响应差异,新方法让所有人公平获益
Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Access
- 设计量化指标与智能转换工具,统一用户提问表达
- 实测显示低素养用户得分比专家低32%,经优化后差距消失
- 适合关注AI公平性、医疗教育等高风险场景的开发者
大型语言模型在医疗、教育和公共服务中日益普及,应确保不同用户无论文笔水平或提问技巧如何,都能获得一致的服务质量。然而现有研究多聚焦于对抗攻击与提示注入,忽视了语义相同但表述不同的请求可能引发不同响应的问题。本文提出“提示特权”(Prompt Privilege)概念:即使意图相同,具备提示工程能力的用户仍能获得更优结果。为此,我们构建统一框架,引入提示公平性评分(PES)量化性能一致性,并提出提示公平变换器(PET),自动将用户原始请求转为语义等价且更易访问的表达形式,实现从用户端到系统端的智能适配。在MedQA基准测试中,低素养用户组与专家组表现存在显著差异(平均分差32%),应用PET后该差距消除,同时保持语义一致性。本工作首次将提示平等性作为关键可衡量维度,推动以系统为中心的公平性建设。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly integrated into healthcare, education, public services, and everyday decision making. They should provide comparable assistance regardless of a user's literacy, communication style, or prompt-engineering expertise. However, existing research on prompt robustness primarily focuses on adversarial attacks, prompt injection, and prompt optimization, while overlooking whether semantically equivalent requests receive different responses simply because they are phrased differently. We refer to this accessibility challenge as "Prompt Privilege": users with greater prompting expertise systematically obtain better model performance despite expressing the same underlying intent. To address this problem, we present a unified framework for measuring and mitigating accessibility disparities in LLM interactions. We introduce Prompt Equity Score (PES), a quantitative metric for evaluating performance consistency across user populations, and Prompt Equity Transformer (PET), an LLM-based agent that automatically transforms user requests into semantically equivalent, accessibility-oriented prompts while preserving their intent. PET shifts prompt optimization from the user to the AI system, functioning as an intelligent accessibility layer between users and foundation models. Experiments on the MedQA benchmark demonstrate measurable prompt privilege, with statistically significant performance disparities between low-literacy and expert-prompting cohorts. Applying PET eliminates these disparities while preserving semantic fidelity, demonstrating that accessibility-oriented prompt normalization can improve equitable AI access. By introducing prompt privilege as a new dimension of AI accessibility and PET as a practical solution, this work advances system-centered accessibility and provides a foundation for more fair, trustworthy, and inclusive AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。