用能量图谱解析大模型响应，定位关键提示句。
Interpreting Black-Box Large Language Models with Sentence-Level Energy Landscapes
- 构建能量模型模拟大模型内部逻辑，生成句子级影响图谱。
- 训练后无需调用API，即可量化各提示句对输出的影响。
- 适用于需解释闭源大模型决策的开发者与研究人员。
由于专有大语言模型（LLM）仅通过封闭API访问，其可解释性严重不足，制约了负责任部署。为此，我们提出一种无需模型结构、基于后处理的句子级归因解释器。该方法训练一个能量基模型（EBM）作为代理，捕捉提示与回复间的内在概念一致性，形成能量景观以指导轻量级解释网络的训练。独特之处在于，解释器可独立运行：训练完成后，无需再次调用原始LLM API，即可量化特定提示句对目标输出的影响。通过在多样化输入上全局训练局部解释器，框架能捕捉更广泛的生成模式，减少个体样本偏差。实验表明，该EBM能准确模拟目标LLM，使解释器有效识别出生成特定输出的关键提示句。
原文摘要 · Abstract (English)
The widespread adoption of proprietary Large Language Models (LLMs) accessed strictly through closed APIs has created a critical challenge for responsible deployment: a fundamental lack of interpretability. To address this, we propose a model-agnostic, post-hoc attribution interpreter operating at the sentence level. Our approach trains an Energy-Based Model (EBM) as a surrogate to capture the LLM's internal conceptual consistency between prompts and responses. This energy landscape guides the training of a lightweight interpreter network. Uniquely, our interpreter operates as a standalone tool; once trained, it quantifies the influence of prompt sentences on a user-specified target output without requiring further API queries to the LLM. By globally training a local interpreter across diverse inputs, our framework captures broader generation patterns and mitigates instance-specific biases. Experiments demonstrate that our EBM accurately simulates the target LLM, allowing the interpreter to effectively identify the prompt sentences most influential in generating specific target outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。