用知识探针揭示大模型是否理解文本背后的隐含知识
Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspective
- 提出KnowProb方法，从后验角度探测模型对隐含知识的理解
- 发现小/大模型仅学单一表征分布，难捕捉文本深层知识
- 提供六种解释路径，帮助研究者可解释地评估模型缺陷
预训练语言模型在大量无标签数据上训练，展现出卓越的推理能力，但其黑箱特性带来的可信度问题日益突出。本文提出一种新型后验解释的知识引导探针方法KnowProb，旨在探测黑箱语言模型是否理解文本之外的隐含知识，而非仅关注表面内容。我们基于文本潜在语义提出六种解释，包括三种知识型理解与三种关联型推理。实验表明，当前小规模（或大规模）语言模型仅学习单一表征分布，在捕捉给定文本背后隐藏知识方面仍面临显著挑战。此外，所提方法能从多个探针视角有效识别现有黑箱模型的局限性，有助于推动可解释性检测研究。
原文摘要 · Abstract (English)
Pre-trained Language Models (PLMs) are trained on large amounts of unlabeled data, yet they exhibit remarkable reasoning skills. However, the trustworthiness challenges posed by these black-box models have become increasingly evident in recent years. To alleviate this problem, this paper proposes a novel Knowledge-guided Probing approach called KnowProb in a post-hoc explanation way, which aims to probe whether black-box PLMs understand implicit knowledge beyond the given text, rather than focusing only on the surface level content of the text. We provide six potential explanations derived from the underlying content of the given text, including three knowledge-based understanding and three association-based reasoning. In experiments, we validate that current small-scale (or large-scale) PLMs only learn a single distribution of representation, and still face significant challenges in capturing the hidden knowledge behind a given text. Furthermore, we demonstrate that our proposed approach is effective for identifying the limitations of existing black-box models from multiple probing perspectives, which facilitates researchers to promote the study of detecting black-box models in an explainable way.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。