EvalxNLP可一键评测NLP模型解释方法的可靠性与易懂性。
EvalxNLP: A Framework for Benchmarking Post-Hoc Explainability Methods on NLP Models
- 整合8种主流可解释性方法，支持多维度评估
- 通过人类评测验证用户满意度，结果良好
- 内置LLM生成文本解释，适合非专家使用
随着自然语言处理（NLP）模型在高风险场景中的广泛应用，其可解释性成为关键挑战。面对多样化的解释方法和不同利益相关者的需求，开发能针对具体场景选择合适解释的框架愈发重要。为此，我们提出EvalxNLP，一个用于基准测试基于Transformer的NLP模型特征归因方法的Python框架。该框架集成来自可解释人工智能（XAI）文献中的八种广泛认可的解释技术，支持基于忠实性、合理性与复杂度等关键属性生成并评估解释。此外，框架提供基于大语言模型（LLM）的交互式文本解释，帮助用户理解生成结果与评估结论。人类评估结果显示用户满意度较高，表明EvalxNLP在不同用户群体中具有良好的适用性。通过提供友好且可扩展的平台，该框架旨在推动可解释性工具的普及，支持NLP领域XAI技术的系统性比较与持续发展。
原文摘要 · Abstract (English)
As Natural Language Processing (NLP) models continue to evolve and become integral to high-stakes applications, ensuring their interpretability remains a critical challenge. Given the growing variety of explainability methods and diverse stakeholder requirements, frameworks that help stakeholders select appropriate explanations tailored to their specific use cases are increasingly important. To address this need, we introduce EvalxNLP, a Python framework for benchmarking state-of-the-art feature attribution methods for transformer-based NLP models. EvalxNLP integrates eight widely recognized explainability techniques from the Explainable AI (XAI) literature, enabling users to generate and evaluate explanations based on key properties such as faithfulness, plausibility, and complexity. Our framework also provides interactive, LLM-based textual explanations, facilitating user understanding of the generated explanations and evaluation outcomes. Human evaluation results indicate high user satisfaction with EvalxNLP, suggesting it is a promising framework for benchmarking explanation methods across diverse user groups. By offering a user-friendly and extensible platform, EvalxNLP aims at democratizing explainability tools and supporting the systematic comparison and advancement of XAI techniques in NLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。