提出可合法违规的智能体认知架构,让AI在紧急时合理打破规则。
PAVE: A Cognitive Architecture for Legitimate Violation in Generative Agent Societies

- 四模块架构:感知情境、评估合法性、决策是否违规、模拟执行
- 实现合法违规、服从权威、范围受限、事后恢复四大特性
- 适用于需应急决策的AI社会,如交通指挥或消防疏散场景
基于大语言模型的生成式智能体能在协作环境中模拟可信的人类行为,但在火灾疏散或上级监督等紧急情况下是否应违反规则,尚缺乏清晰机制。本文提出PAVE(感知、评估、裁决、模拟)四模块认知架构,端到端解决此问题:(i) 感知模块提取包含明确权威距离、同伴行为及严重性标记的情境线索;(ii) 评估模块沿五个维度打分,包含显式的合法性判断,检验必要性、比例性和替代方案缺失;(iii) 裁决模块在严格合法性门控下决定遵从或违规,阈值由个体人格设定;(iv) 模拟模块执行裁决,并将违规限定在触发规则范围内。我们在基于Smallville改造的Voville(基于瓦片的交通环境)中实例化PAVE,评估涵盖三种场景、四种LLM主干模型及聚焦消融实验。PAVE智能体同时满足四个性质:合法违规(仅当触发条件成立)、权威服从(执法者指令优先于高合法性)、范围受限(违规仅限目标规则)、恢复能力(触发结束后恢复基线)。相比原始模型,PAVE智能体在四项指标上表现更结构化、可解释,人类评估者认为其行为更可信。移除合法性门控后重现原始模型的失败模式。我们开源了Voville、PAVE提示词与代码及评估流水线。
原文摘要 · Abstract (English)
Generative agents based on large language models reproduce believable human behavior in cooperative settings, but how they should reason in situations where rule-breaking may be required, such as fire evacuation or authority-supervised emergency, remains poorly characterized. We propose PAVE (Perception, Assessment, Verdict, Emulation), a novel four-module cognitive architecture that addresses this gap end to end: (i) Perception extracts a structured context with explicit authority distance, peer behaviors, and severity-tagged situational cues; (ii) Assessment scores the context along five scalars including an explicit legitimacy judgment that checks necessity, proportionality, and absence of alternatives; (iii) Verdict decides to comply or violate under a hard legitimacy gate, with a per-agent threshold elicited from the persona; (iv) Emulation enacts the verdict and scopes the violation to the rule the trigger justifies. We instantiate PAVE in Voville, a tile-based traffic environment forked from Smallville, and evaluate across three scenarios, four LLM backbones, and a focused ablation. PAVE agents satisfy four properties simultaneously: legitimate violation (only when a trigger justifies it), authority deference (officer instructions override even high legitimacy), bounded scope (violations confined to the targeted rule), and recovery (baseline restored once the trigger ends). PAVE agents make more structured and interpretable decisions than vanilla across all four properties, and human evaluators rate them as more plausible. Ablating the legitimacy gate reproduces vanilla-like failures. We release Voville, the PAVE prompts and code, and the evaluation pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。