开源工具包，一键解析大模型输出背后的推理逻辑。
ICX360: In-Context eXplainability 360 Toolkit
- 基于上下文提示，结合扰动与梯度方法解释LLM决策
- 支持检索增强生成、自然语言生成等场景的可解释性分析
- 适合关注大模型可信性与安全性的研究人员与开发者
大型语言模型（LLMs）已广泛应用于日常生活，并逐步进入会议摘要、医疗问答等高风险领域。如同早期预测模型，亟需开发解释工具以理解其输出——无论是摘要、列表还是问题回答。为此，我们推出面向上下文提示的可解释性工具包ICX360（In-Context Explainability 360），一个开源Python工具集，专注于用户提供的上下文（或一般提示）对LLM输出的影响。ICX360集成三种近期提出的可解释性方法，涵盖黑盒（通过扰动）与白盒（通过梯度）技术。该工具包可通过https://github.com/IBM/ICX360获取，提供快速入门指南与详细教程，覆盖检索增强生成、自然语言生成及对抗攻击（jailbreaking）等典型应用场景。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have become ubiquitous in everyday life and are entering higher-stakes applications ranging from summarizing meeting transcripts to answering doctors' questions. As was the case with earlier predictive models, it is crucial that we develop tools for explaining the output of LLMs, be it a summary, list, response to a question, etc. With these needs in mind, we introduce In-Context Explainability 360 (ICX360), an open-source Python toolkit for explaining LLMs with a focus on the user-provided context (or prompts in general) that are fed to the LLMs. ICX360 contains implementations for three recent tools that explain LLMs using both black-box and white-box methods (via perturbations and gradients respectively). The toolkit, available at https://github.com/IBM/ICX360, contains quick-start guidance materials as well as detailed tutorials covering use cases such as retrieval augmented generation, natural language generation, and jailbreaking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。