arXiv:2604.19663cs.IRcs.LG2026-04

系统复现11种推荐解释方法,揭示其真实效果与适用边界。

From Top-1 to Top-K: A Reproducibility Study and Benchmarking of Counterfactual Explanations for Recommender Systems

  • 统一框架对比不同解释方法,涵盖显式/隐式格式、项级/列表级评估。
  • 发现解释有效性与稀疏性权衡依赖具体方法和设置,尤其显式格式下更明显。
  • 多模型多数据验证,揭示图模型解释器在大规模推荐系统中扩展性不足。

反事实解释(CE)通过识别最小用户-物品交互修改来直观理解推荐系统结果。然而,现有推荐系统中的CE方法评价标准不一,涉及不同数据集、推荐模型、评估指标与解释格式,导致可复现性差、比较困难。本文系统复现、重实现并重新评估了11种最先进的推荐系统反事实解释方法,涵盖原生解释器(如LIME-RS、SHAP、PRINCE、ACCENT、LXR、GREASE)及专为GNN设计的图结构解释器。提出统一基准框架,从解释格式(隐式vs显式)、评估层级(项级vs列表级)、扰动范围(用户交互向量vs用户-物品交互图)三方面评估。评估包含有效性、稀疏性与计算复杂度指标,并将项级评估扩展至顶K列表级。在三个真实世界数据集和六种代表性推荐模型上进行大量实验,分析先前报告的性能优势是否在多样设置下泛化。结果显示:有效性与稀疏性权衡强烈依赖方法与评估设置,尤其显式格式下;解释性能在项级与列表级评估间基本一致;多个图基解释器在大规模推荐图上存在显著可扩展性问题。研究结果修正并挑战了此前对推荐系统反事实生成方法鲁棒性与实用性的结论。项目代码已开源:https://github.com/L2R-UET/CFExpRec。

原文摘要 · Abstract (English)

Counterfactual explanations (CEs) provide an intuitive way to understand recommender systems by identifying minimal modifications to user-item interactions that alter recommendation outcomes. Existing CE methods for recommender systems, however, have been evaluated under heterogeneous protocols, using different datasets, recommenders, metrics, and even explanation formats, which hampers reproducibility and fair comparison. Our paper systematically reproduces, re-implement, and re-evaluate eleven state-of-the-art CE methods for recommender systems, covering both native explainers (e.g., LIME-RS, SHAP, PRINCE, ACCENT, LXR, GREASE) and specific graph-based explainers originally proposed for GNNs. Here, a unified benchmarking framework is proposed to assess explainers along three dimensions: explanation format (implicit vs. explicit), evaluation level (item-level vs. list-level), and perturbation scope (user interaction vectors vs. user-item interaction graphs). Our evaluation protocol includes effectiveness, sparsity, and computational complexity metrics, and extends existing item-level assessments to top-K list-level explanations. Through extensive experiments on three real-world datasets and six representative recommender models, we analyze how well previously reported strengths of CE methods generalize across diverse setups. We observe that the trade-off between effectiveness and sparsity depends strongly on the specific method and evaluation setting, particularly under the explicit format; in addition, explainer performance remains largely consistent across item level and list level evaluations, and several graph-based explainers exhibit notable scalability limitations on large recommender graphs. Our results refine and challenge earlier conclusions about the robustness and practicality of CE generation methods in recommender systems: https://github.com/L2R-UET/CFExpRec.

推荐系统反事实解释可复现性基准测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。