通过激活梯度追踪，精准定位大模型生成有害内容的根源。
Where Did It Go Wrong? Attributing Undesirable LLM Behaviors via Representation Gradient Tracing
- 在模型激活空间中分析表示梯度，实现语义可解释的归因。
- 可精确定位导致错误输出的具体训练样本和关键词句。
- 适用于检测有害内容、后门攻击和知识污染，适合安全审计者使用。
大型语言模型虽表现出色，但常产生有害内容、事实错误与社会偏见，其故障根因诊断面临重大挑战。现有基于参数梯度的归因方法因信号噪声大、计算复杂而效果不佳。本文提出一种新框架，直接在模型激活空间中分析表示及其梯度，提供语义明确的信号，将输出与训练数据关联。系统评估显示，该方法不仅在样本级归因上表现优异，还能实现细粒度的词元级分析，精准识别出因果影响模型行为的具体训练样本和短语。本工作为理解、审计和缓解大模型风险提供了强大工具。代码已开源：https://github.com/plumprc/RepT。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated remarkable capabilities, yet their deployment is frequently undermined by undesirable behaviors such as generating harmful content, factual inaccuracies, and societal biases. Diagnosing the root causes of these failures poses a critical challenge for AI safety. Existing attribution methods, particularly those based on parameter gradients, often fall short due to prohibitive noisy signals and computational complexity. In this work, we introduce a novel and efficient framework that diagnoses a range of undesirable LLM behaviors by analyzing representation and its gradients, which operates directly in the model's activation space to provide a semantically meaningful signal linking outputs to their training data. We systematically evaluate our method for tasks that include tracking harmful content, detecting backdoor poisoning, and identifying knowledge contamination. The results demonstrate that our approach not only excels at sample-level attribution but also enables fine-grained token-level analysis, precisely identifying the specific samples and phrases that causally influence model behavior. This work provides a powerful diagnostic tool to understand, audit, and ultimately mitigate the risks associated with LLMs. The code is available at https://github.com/plumprc/RepT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。