用因果分析检测大模型说谎、偏见等错误行为，效果更好更轻量。
LLMScan: Causal Scan for LLM Misbehavior Detection
- 基于因果推断分析输入词元和注意力层的作用
- 不同行为下因果分布差异明显，可精准识别异常
- 适合需要高可靠性的大模型应用，如医疗、金融
尽管大语言模型在多个领域取得成功，但其生成虚假、偏见或有害内容的潜在风险仍不容忽视，尤其在关键应用场景中。这迫切需要系统性方法来检测和防范此类错误行为。现有方法多针对特定问题，本文提出一种基于因果分析的新型监控技术LLMScan，从因果推断视角系统监测大模型内部运作机制。该方法假设模型在错误行为时其内部‘大脑’表现不同，通过分析输入词元和Transformer层的因果贡献，有效检测异常行为。大量实验表明，在多种任务和模型上，正常行为与错误行为的因果分布存在显著差异，从而可构建准确且轻量的检测器，适用于多种错误行为检测任务。
原文摘要 · Abstract (English)
Despite the success of Large Language Models (LLMs) across various fields, their potential to generate untruthful, biased and harmful responses poses significant risks, particularly in critical applications. This highlights the urgent need for systematic methods to detect and prevent such misbehavior. While existing approaches target specific issues such as harmful responses, this work introduces LLMScan, an innovative LLM monitoring technique based on causality analysis, offering a comprehensive solution. LLMScan systematically monitors the inner workings of an LLM through the lens of causal inference, operating on the premise that the LLM's `brain' behaves differently when misbehaving. By analyzing the causal contributions of the LLM's input tokens and transformer layers, LLMScan effectively detects misbehavior. Extensive experiments across various tasks and models reveal clear distinctions in the causal distributions between normal behavior and misbehavior, enabling the development of accurate, lightweight detectors for a variety of misbehavior detection tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。