测试大模型对用户身份设定的敏感度，发现真实对话中身份信息会影响回答准确性。
Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering
- 设计真实用户角色线索，评估大模型在不同身份背景下的回答稳定性。
- 部分模型因用户身份设定出现拒绝回答或编造限制，准确率下降明显。
- 适合关注模型安全与可信性的研究人员和开发者使用。
大型语言模型应在事实问答中始终基于客观知识提供真实答案，不受用户上下文（如自我披露的个人信息或系统个性化）影响。本文首次系统评估了大模型对询问者人格（inquiry personas）的鲁棒性，即用户在真实交互中透露的身份、专业性或信念等属性。以往研究多关注对抗性输入或干扰项，而本文聚焦于人类中心化的自然角色线索。实验发现，这些线索会显著改变问答准确率，并引发拒绝回答、虚构限制和角色混淆等失效模式。这表明模型对用户表述方式的敏感性可能损害其事实可靠性，也证明询问者人格测试是评估模型鲁棒性的有效手段。
原文摘要 · Abstract (English)
Large Language Models (LLMs) should answer factual questions truthfully, grounded in objective knowledge, regardless of user context such as self-disclosed personal information, or system personalization. In this paper, we present the first systematic evaluation of LLM robustness to inquiry personas, i.e. user profiles that convey attributes like identity, expertise, or belief. While prior work has primarily focused on adversarial inputs or distractors for robustness testing, we evaluate plausible, human-centered inquiry persona cues that users disclose in real-world interactions. We find that such cues can meaningfully alter QA accuracy and trigger failure modes such as refusals, hallucinated limitations, and role confusion. These effects highlight how model sensitivity to user framing can compromise factual reliability, and position inquiry persona testing as an effective tool for robustness evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。