提出可解释的句向量模型PropNet，模拟人类认知推理过程。
PropNet: a White-Box and Human-Like Network for Sentence Representation
- 基于句子中的命题构建分层网络，实现完全白盒化表示
- 在语义文本相似度任务上与顶尖模型仍有差距
- 能分析人类对相似度判断的认知机制，适合可解释性研究
近年来，基于Transformer的嵌入方法在句向量表示领域占据主导地位。尽管其在语义文本相似度（STS）等自然语言处理任务中表现优异，但其黑箱特性及依赖大规模数据训练的模式引发了对偏见、可信度和安全性的担忧。尽管已有诸多工作致力于提升嵌入模型的可解释性，但根本问题仍未解决。为实现内在可解释性，我们提出一种纯粹白盒且类人化的句向量表示网络——PropNet。受认知科学发现启发，PropNet基于句子所包含的命题构建层次化网络。实验表明，PropNet在STS任务上与当前最先进（SOTA）模型相比仍存在显著差距，案例研究揭示了巨大的改进空间。此外，PropNet使我们能够分析和理解STS基准背后的人类认知过程。
原文摘要 · Abstract (English)
Transformer-based embedding methods have dominated the field of sentence representation in recent years. Although they have achieved remarkable performance on NLP missions, such as semantic textual similarity (STS) tasks, their black-box nature and large-data-driven training style have raised concerns, including issues related to bias, trust, and safety. Many efforts have been made to improve the interpretability of embedding models, but these problems have not been fundamentally resolved. To achieve inherent interpretability, we propose a purely white-box and human-like sentence representation network, PropNet. Inspired by findings from cognitive science, PropNet constructs a hierarchical network based on the propositions contained in a sentence. While experiments indicate that PropNet has a significant gap compared to state-of-the-art (SOTA) embedding models in STS tasks, case studies reveal substantial room for improvement. Additionally, PropNet enables us to analyze and understand the human cognitive processes underlying STS benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。