首个统一评估框架,让反事实解释方法公平比拼。
CEL: Comprehensive Counterfactual Explanations Library and Benchmark

- 构建18个数据集+14种方法的统一库与评测标准
- 覆盖有效性、稀疏性、合理性等多维度指标
- 适合研究可解释AI的学者和开发者使用
反事实解释是可解释人工智能中的重要方法,能给出改变输入以达成期望结果的具体操作。尽管近年方法已考虑稀疏性、可行动性和合理性等特性,但公平系统评估仍具挑战。现有研究常使用不同数据划分、模型和评估指标,难以客观比较。为此,我们提出CEL(反事实解释库与基准),一个统一的库与评测平台,包含18个不同规模与复杂度的数据集,以及14种常用反事实方法的实现或重实现。在标准化设置下,我们在多种数据集上对各类方法进行了全面定量比较,评估协议涵盖有效性、覆盖率、稀疏性、接近性及分布合理性等多个互补指标,包括基于密度和异常值的度量以评估生成样本的真实性。据我们所知,这是首个在统一可复现框架中系统评估近期反事实解释方法的综合性基准。相比已有工作,当前平台更完整、更新且具备一致评估流程,旨在提升可复现性,实现公平比较,并为未来方法开发提供基础工作台。
原文摘要 · Abstract (English)
Counterfactual explanations are a prominent approach in explainable artificial intelligence (xAI), providing actionable guidance on what input changes would alter a model's prediction to a desired outcome. While early methods primarily focused on minimal feature changes, recent work incorporates additional properties such as sparsity, actionability and plausibility. Despite this progress, fair and systematic evaluation remains challenging. Existing studies often rely on different data splits, predictive models, and evaluation metrics, which limits objective comparison across methods. To fill this gap, we introduce CEL (Counterfactual Explanations Library), a unified library and benchmark for counterfactual explanations designed to support consistent implementation and evaluation. CEL includes 18 datasets of varying size and complexity and provides implementations or reimplementations of 14 widely used counterfactual methods. Using this standardized setup, we conduct a comprehensive quantitative comparison across a variety of methods on datasets that differ in size, number, and types of attributes. The evaluation protocol incorporates multiple complementary metrics capturing validity, coverage, sparsity, proximity, and distributional plausibility, including density- and outlier-based measures to assess the realism of generated counterfactuals. To the best of our knowledge, this is the first comprehensive benchmark that systematically evaluates recent counterfactual explanation methods within a unified and reproducible framework. While prior libraries and benchmarking efforts exist in the literature, many are outdated, limited in scope, or lack consistent evaluation protocols. The proposed benchmark aims to improve reproducibility, enable fair comparison, and establish a workbench for the development of future counterfactual explanation methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。