提出可完美还原分类模型决策的词嵌入解释方法，用于科研影响力预测。
Explaining word embeddings with perfect fidelity: Case study in research impact prediction
- 基于逻辑回归模型设计自评实体法（SMER），直接关联词嵌入与预测结果
- 在CORD-19 5万篇论文上验证，比LIME、SHAP等方法解释更准确
- 适合需高可信度解释的科研评价、学术分析场景
当前最优的学术文档质量预测方法依赖于嵌入模型。这些模型不仅能对训练数据中出现过的词进行预测，还能处理未见词汇，这对不断演化的研究术语至关重要。尽管模型无关的解释方法如LIME和SHAP可用于解释基于嵌入的分类器，但其解释结果与模型真实决策存在偏差。本文提出一种新的特征重要性方法——自评实体法（SMER），适用于基于词嵌入的逻辑回归分类模型。我们证明，SMER具有理论上的完美保真度：单个词的SMER得分的平均值恰好等于模型预测的对数几率（logit）。通过在包含50,000篇研究论文的CORD-19语料库上开展五项多样化实验，我们使用AOPC曲线分析证实，SMER的解释效果优于LIME、SHAP及全局树代理模型。
原文摘要 · Abstract (English)
The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-model Rated Entities (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. Through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。