用语义概念解释大模型输出,可查偏见、调方向,无需重训练。
Concept-Level Explainability for Auditing & Steering LLM Responses
- 基于提示中语义概念的权重评估,而非单个词元。
- 提升情感转向0.252,攻击成功率从46.3%降至24.2%。
- 适合做安全审计和无须重训练的响应引导。
随着大语言模型广泛应用,其安全性和对齐性问题日益突出。通过识别提示中影响输出特定方面的部分来调节模型行为是一种有效方法。尽管词元级归因方法有潜力,但在文本生成中仍存在局限:它们分别解释每个词元的出现,而非整个输出的语义。我们提出ConceptX,一种与模型无关的、基于概念级别的可解释性方法,能识别提示中的语义丰富词元(即概念),并根据输出语义相似度分配重要性。不同于现有词元级方法,ConceptX支持原位词元替换以保持上下文完整性,并可灵活适配不同解释目标,如性别偏见。该方法既可用于审计(揭示偏见来源),也可用于引导(修改提示以改变情感或降低有害性),且无需重新训练。在三个大语言模型上,ConceptX在忠实度和人类一致性方面均优于TokenSHAP等词元级方法。在引导任务中,情感转变提升至0.252(随机编辑为0.131),攻击成功率从0.463降至0.242,优于归因和改写基线。尽管提示工程与自解释方法有时也能生成更安全输出,ConceptX提供了透明且可信的替代方案,证明了基于归因的可解释性在引导大模型行为中的实用价值。
原文摘要 · Abstract (English)
As large language models (LLMs) become widely deployed, concerns about their safety and alignment grow. An approach to steer LLM behavior, such as mitigating biases or defending against jailbreaks, is to identify which parts of a prompt influence specific aspects of the model's output. Token-level attribution methods offer a promising solution, but still struggle in text generation, explaining the presence of each token in the output separately, rather than the underlying semantics of the entire LLM response. We introduce ConceptX, a model-agnostic, concept-level explainability method that identifies the concepts, i.e., semantically rich tokens in the prompt, and assigns them importance based on the outputs' semantic similarity. Unlike current token-level methods, ConceptX also offers to preserve context integrity through in-place token replacements and supports flexible explanation goals, e.g., gender bias. ConceptX enables both auditing, by uncovering sources of bias, and steering, by modifying prompts to shift the sentiment or reduce the harmfulness of LLM responses, without requiring retraining. Across three LLMs, ConceptX outperforms token-level methods like TokenSHAP in both faithfulness and human alignment. Steering tasks boost sentiment shift by 0.252 versus 0.131 for random edits and lower attack success rates from 0.463 to 0.242, outperforming attribution and paraphrasing baselines. While prompt engineering and self-explaining methods sometimes yield safer responses, ConceptX offers a transparent and faithful alternative for improving LLM safety and alignment, demonstrating the practical value of attribution-based explainability in guiding LLM behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。