arXiv:2604.09443cs.CLcs.AI2026-04被引 6

提出多层级指令体系,解决大模型代理中复杂指令冲突问题

Many-Tier Instruction Hierarchy in LLM Agents

  • 设计可扩展的多层级指令优先级机制,支持任意数量的权限层级
  • 在包含12级冲突指令的基准上,前沿模型准确率仅约40%
  • 适用于需要高可靠性决策的复杂代理系统,如自动化任务执行

大型语言模型代理从多种来源接收指令——系统消息、用户提示、工具输出、其他代理等——每种指令具有不同的可信度和权威性。当指令冲突时,代理必须可靠地遵循最高权限指令以确保安全与高效。当前主流的指令层级(IH)方法假设仅有少量固定权限等级(通常少于五级),并依赖严格的角色标签(如系统 > 用户),这在真实代理场景中难以应对跨源、跨上下文的复杂冲突。本文提出多层级指令层级(ManyIH),一种可处理任意多权限层级指令冲突的范式。我们构建了首个针对ManyIH的基准测试ManyIH-Bench,要求模型在最多12个不同权限层级的冲突指令中进行判断,包含853个代理任务(427个编程任务,426个指令遵循任务)。该基准通过大模型生成并经人工验证的约束条件,构建出涵盖46个真实代理的现实且困难的测试案例。实验表明,即使最先进的模型在指令冲突规模扩大时表现仍差(准确率约40%)。本工作凸显了在代理环境中显式处理细粒度、可扩展指令冲突的迫切需求。

原文摘要 · Abstract (English)

Large language model agents receive instructions from many sources-system messages, user prompts, tool outputs, other agents, and more-each carrying different levels of trust and authority. When these instructions conflict, agents must reliably follow the highest-privilege instruction to remain safe and effective. The dominant paradigm, instruction hierarchy (IH), assumes a fixed, small set of privilege levels (typically fewer than five) defined by rigid role labels (e.g., system > user). This is inadequate for real-world agentic settings, where conflicts can arise across far more sources and contexts. In this work, we propose Many-Tier Instruction Hierarchy (ManyIH), a paradigm for resolving instruction conflicts among instructions with arbitrarily many privilege levels. We introduce ManyIH-Bench, the first benchmark for ManyIH. ManyIH-Bench requires models to navigate up to 12 levels of conflicting instructions with varying privileges, comprising 853 agentic tasks (427 coding and 426 instruction-following). ManyIH-Bench composes constraints developed by LLMs and verified by humans to create realistic and difficult test cases spanning 46 real-world agents. Our experiments show that even the current frontier models perform poorly (~40% accuracy) when instruction conflict scales. This work underscores the urgent need for methods that explicitly target fine-grained, scalable instruction conflict resolution in agentic settings.

指令理解智能代理多层级

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。