让大模型代理通过反馈迭代修正政策理解，减少因规则模糊导致的错误行为。
PolicyBank: Evolving Policy Understanding for LLM Agents
- 构建可迭代更新的政策记忆库，动态修正对规则的理解
- 在存在政策漏洞的场景中，成功率提升至接近人类水平的82%
- 适合需严格遵守复杂规则的自动化系统开发者
在组织政策约束下运行的大语言模型代理，必须遵守通常以自然语言描述的授权规则。但实际规则常存在歧义或逻辑语义漏洞，导致代理行为系统性偏离真实要求。本文提出PolicyBank，一种记忆机制，能维护分工具级别的政策洞察，并通过交互与纠错反馈持续优化理解。不同于现有记忆机制将政策视为不可更改的真理，从而固化“合规但错误”的行为，PolicyBank能主动修正误解。我们还构建了一个系统化测试基准，通过扩展流行的工具调用基准，在其中引入可控的政策漏洞，以分离对齐失败与执行失败。实验表明，现有记忆机制在政策漏洞场景下的成功率几乎为零，而PolicyBank可将差距缩小达82%（接近人类基准）。
原文摘要 · Abstract (English)
LLM agents operating under organizational policies must comply with authorization constraints typically specified in natural language. In practice, such specifications inevitably contain ambiguities and logical or semantic gaps that cause the agent's behavior to systematically diverge from the true requirements. We ask: by letting an agent evolve its policy understanding through interaction and corrective feedback from pre-deployment testing, can it autonomously refine its interpretation to close specification gaps? We propose PolicyBank, a memory mechanism that maintains structured, tool-level policy insights and iteratively refines them -- unlike existing memory mechanisms that treat the policy as immutable ground truth, reinforcing "compliant but wrong" behaviors. We also contribute a systematic testbed by extending a popular tool-calling benchmark with controlled policy gaps that isolate alignment failures from execution failures. While existing memory mechanisms achieve near-zero success on policy-gap scenarios, PolicyBank closes up to 82% of the gap toward a human oracle.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。