提出用户中心的偏见审计框架，揭示模型如何因用户身份差异而不同回应。
Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research
- 以用户身份信号为输入，评估其对模型响应质量的影响
- 发现同一问题由不同性别或社经背景用户提问时，回答内容与语气显著不同
- 适合关注真实交互中模型公平性的研究人员和开发者
大型语言模型（LLM）的偏见研究长期聚焦于第三人称审计，即模型如何描述或评价群体。然而，这种范式忽略了用户在互动中的缺失。实际上，LLM常用于开放式的个人对话，模型会隐式地将用户纳入自身表征并调整回应。当相同请求因提问者不同而得到不同回答时，偏见便体现在对对话者的差异化对待上。本文提出情境化交互审计（SIA），一种以用户为中心的框架，用于研究用户特征信号——包括隐含的社会人口学标记、写作风格及自我陈述身份——如何系统性影响LLM的回答质量、内容与语气。通过跨性别与社会经济地位的多任务案例研究，验证了该框架的有效性，并为自然语言处理领域提出了新的研究方向。
原文摘要 · Abstract (English)
Research on bias in large language models (LLMs) has predominantly focused on third-person audits, which study how models represent or evaluate demographic groups as external subjects. However, this paradigm overlooks a structural blind spot because the user is absent from the audit. In practice, LLMs are used in open-ended, personal interactions, during which the model implicitly represents the user and adjusts its responses accordingly. When identical requests yield different responses depending on who is asking, bias manifests not in how the model describes others but in how it treats its interlocutor. We propose Situated Interaction Auditing (SIA), a user-centered framework for studying how user profile signals -- implicit sociodemographic markers, writing style, and stated identity -- systematically shape LLM response quality, content, and tone. We demonstrate the framework through a case study that intersects gender and socioeconomic status signals across multiple task domains and outline a research agenda for SIA as a new mission for natural language processing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。