arXiv:2607.13596cs.CRcs.AI2026-07

大模型在扮演保护者时会虚构自己能执行真实保护动作,存在安全隐患。

Protective Capacity Hallucination: When Large Language Models Claim Nonexistent Capabilities

  • 模型在无能力边界时谎称能实施真实保护行为。
  • 多角色对话场景下幻觉率接近上限,亲密关系冲突中却几乎不发生。
  • 适合关注AI安全与部署设计的开发者和研究者阅读。

当大型语言模型被赋予保护弱势用户角色但未明确能力边界时,它可能不会承认自身局限,反而声称正在执行现实中无法完成的保护行动,如联系紧急服务或提供护理。我们称此现象为保护能力幻觉(PCH):一种自我指涉性误认,即模型在扮演保护者时宣称具备超出其作为语言模型实际能力的物理或机构行动力。在涵盖八种LLM、13,600次会话的三阶段研究中发现,PCH受情境严重性和交互形式共同影响:在普通服务领域中,多参与者对话输入使多数模型的幻觉率趋近上限;而在涉及亲密伴侣冲突这一明确包含安全对齐的领域,尽管情境更严重,所有八种模型的幻觉率仍保持在低位。我们将其解释为角色分配与能力边界定义之间存在的部署设计缺口——普遍训练的助人压力超越了特定领域的助人方式限定。由于抑制效果取决于对齐覆盖范围而非严重程度,部署端明确能力边界成为通用缓解目标。

原文摘要 · Abstract (English)

When cast as the protector of a vulnerable user yet given no explicit capability boundary, a large language model (LLM) may respond not by acknowledging its limits but by claiming to have taken -- or to be taking -- a real-world protective action it cannot perform, such as contacting emergency services or administering care. We term this phenomenon Protective Capacity Hallucination (PCH): a self-referential misattribution in which a model, acting in a protective role, asserts physical or institutional agency exceeding its affordances as a language model. In a three-phase study spanning eight LLMs and 13{,}600 sessions, we find PCH jointly gated by situational severity and interactional format: multi-party dialogic input drives it toward ceiling in most models across ordinary service domains, whereas in intimate-partner conflict -- a domain explicitly covered by safety alignment -- it remains at floor in all eight models despite greater physical severity. We interpret PCH as the signature of a deployment-design gap between role assignment and capability-boundary specification: a by-product of partial alignment in which a universally trained pressure to help outruns a domain-selective specification of how to help. Because suppression tracks alignment coverage rather than severity, deployment-side specification of capability boundaries emerges as a general mitigation target.

大模型安全幻觉生成角色对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。