用因果图提升大模型代理的安全性，预防错误决策。
Enhancing LLM Agent Safety via Causal Influence Prompting
- 构建因果影响图描述任务中因果关系，指导安全决策。
- 在代码执行与手机控制任务中，显著降低有害行为发生率。
- 适合关注大模型代理安全性的研究人员和开发者。
随着由大型语言模型（LLMs）驱动的自主代理在各类辅助任务中展现出潜力，确保其安全可靠的行为对于防止意外后果至关重要。本文提出一种新方法CIP，利用因果影响图（CIDs）识别并缓解代理决策过程中产生的风险。CIDs提供因果关系的结构化表示，使代理能够预判有害结果并做出更安全的决策。该方法包含三个关键步骤：(1) 基于任务说明初始化CID，明确决策流程；(2) 使用CID引导代理与环境交互；(3) 根据观察到的行为和结果迭代优化CID。实验结果表明，该方法在代码执行和移动设备控制任务中均有效提升了安全性。
原文摘要 · Abstract (English)
As autonomous agents powered by large language models (LLMs) continue to demonstrate potential across various assistive tasks, ensuring their safe and reliable behavior is crucial for preventing unintended consequences. In this work, we introduce CIP, a novel technique that leverages causal influence diagrams (CIDs) to identify and mitigate risks arising from agent decision-making. CIDs provide a structured representation of cause-and-effect relationships, enabling agents to anticipate harmful outcomes and make safer decisions. Our approach consists of three key steps: (1) initializing a CID based on task specifications to outline the decision-making process, (2) guiding agent interactions with the environment using the CID, and (3) iteratively refining the CID based on observed behaviors and outcomes. Experimental results demonstrate that our method effectively enhances safety in both code execution and mobile device control tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。