arXiv:2505.00626cs.CLcs.AI2025-05ICML被引 9

发现大模型角色识别的隐藏捷径,并提出机制级修复方法

The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)

  • 通过控制实验发现模型依赖任务类型和文本位置判断角色
  • 数据增强只能缓解问题,无法根治角色混淆缺陷
  • 调整位置编码信号可提升角色区分能力,适合安全敏感场景

集成多角色输入(如系统指令、用户查询、外部工具输出)的大语言模型日益普遍。确保模型准确区分不同角色信息——即我们称为“角色分离”的能力——对保持多角色行为一致性至关重要。尽管近期工作聚焦于顶级提示注入防御,但尚不清楚这些方法是否真正教会模型区分角色,还是仅记忆已知触发词。本文研究“角色分离学习”:如何使大模型稳健区分系统与用户标记。通过一个简单、可控的实验框架,我们发现微调后的模型常依赖两个代理信号进行角色识别:(1) 任务类型利用,(2) 与文本起始位置的接近程度。虽然数据增强能部分缓解这些捷径,但通常导致反复修补而非根本解决。为此,我们提出强化标记角色边界的不变信号,通过调整输入编码中的逐令牌提示。特别是,操纵位置ID有助于模型学习更清晰的角色区分,减少对表面代理的依赖。从机制中心视角出发,本工作揭示了大模型实现可靠多角色行为的新路径,避免仅靠记忆已知提示或触发词。

原文摘要 · Abstract (English)

Large language models (LLMs) that integrate multiple input roles (e.g., system instructions, user queries, external tool outputs) are increasingly prevalent in practice. Ensuring that the model accurately distinguishes messages from each role -- a concept we call \emph{role separation} -- is crucial for consistent multi-role behavior. Although recent work often targets state-of-the-art prompt injection defenses, it remains unclear whether such methods truly teach LLMs to differentiate roles or merely memorize known triggers. In this paper, we examine \emph{role-separation learning}: the process of teaching LLMs to robustly distinguish system and user tokens. Through a \emph{simple, controlled experimental framework}, we find that fine-tuned models often rely on two proxies for role identification: (1) task type exploitation, and (2) proximity to begin-of-text. Although data augmentation can partially mitigate these shortcuts, it generally leads to iterative patching rather than a deeper fix. To address this, we propose reinforcing \emph{invariant signals} that mark role boundaries by adjusting token-wise cues in the model's input encoding. In particular, manipulating position IDs helps the model learn clearer distinctions and reduces reliance on superficial proxies. By focusing on this mechanism-centered perspective, our work illuminates how LLMs can more reliably maintain consistent multi-role behavior without merely memorizing known prompts or triggers.

大模型角色分离安全提示工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。