模型看似公平，实则依赖提示线索；隐藏线索后偏见上升4.4个百分点。
Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues
- 通过改变身份提示方式，测试模型在不同线索下的真实公平性。
- 隐藏显式标签后，有害决策增加4.4个百分点，公平性显著下降。
- 提出可适配现有评估的鲁棒性指标，识别表面合规与真实道德安全。
随着大语言模型在医疗、法律和招聘等关键领域承担道德责任，必须检验其伦理行为是真实的还是表面的。我们发现当前公平性评估严重高估了道德安全性：当人口属性以显式标签形式呈现时，模型表现公平；但当同一属性需被推断时，公平性显著下降。我们将此现象称为‘表演式合规’——模型仅在提示形式类似公平评估时才表现公正。我们提出一种提示变异方法，在保持道德困境和身份不变的前提下，仅改变身份的表达方式。隐藏显式标签后，有害决策上升4.4个百分点，模型安全排名发生变化，且该趋势在模型正确推断身份时仍持续存在，排除了误判可能。我们引入‘提示可见性差距’（Cue Visibility Gap）这一模型无关的鲁棒性度量，可嵌入任何现有公平性基准，区分真实与表演式道德安全。忽略提示变异的评估仅衡量表面合规，而非道德鲁棒性，不应用于高风险场景的部署决策。
原文摘要 · Abstract (English)
As large language models take on morally consequential roles in healthcare, legal, and hiring contexts, we need to examine whether their ethical behaviors are genuine or superficial. We show that current fairness evaluations substantially overestimate moral safety. Models appear fair when demographic identity is stated as an explicit label, yet become measurably less fair when the same identity must be inferred. We term this failure performative compliance, where a model is fair when the presentation resembles a fairness evaluation and less fair as that cue weakens. We introduce a cue-variation methodology that holds the moral dilemma and the demographic identity fixed and varies only how that identity is conveyed. Hiding the explicit label raises harmful decisions by +4.4 pp, changes model safety rankings, and the shift persists when models correctly infer the demographic, ruling out attribution error. We propose the Cue Visibility Gap, a model-agnostic robustness metric that can be added to any existing fairness benchmark to separate genuine from performative moral safety. Fairness evaluations that omit cue variation measure surface compliance, not moral robustness, and should not ground deployment decisions in high-stakes settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。