用双语图神经网络检测环保声明,参数量少30倍却更准。
Efficient Environmental Claim Detection with Hyperbolic Graph Neural Networks
- 将声明句转为语法依赖图,用词向量与词性标签作节点特征
- 超球面图网络在环保声明检测上性能超越主流模型,参数少30倍
- 特别适合算力有限的开源项目,处理层级结构数据优势明显
基于Transformer的模型,尤其是大语言模型(LLMs)在自然语言处理中占据主导地位,广泛应用于文本生成、摘要和假新闻检测等任务。这些模型虽部署简便且可靠,但训练和推理需要大量计算资源,限制了其在资源受限场景中的应用,尤其在开源社区中计算能力普遍匮乏。本文提出一种基于图神经网络(GNNs)与超球面图神经网络(HGNNs)的环保声明检测方法,将任务重构为图分类问题:将声明句子转化为依存句法图,使用word2vec与可学习词性(POS)标签嵌入作为节点特征,并在边关系中编码句法依赖。实验表明,该方法特别是庞加莱空间中的超球面图网络(P-HGNNs),在环保声明检测任务上性能优于当前最优模型,同时参数量减少最多达30倍。结果还显示,显式建模层次化(树状)结构使HGNN显著优于欧氏空间中的同类模型。
原文摘要 · Abstract (English)
Transformer based models, especially large language models (LLMs) dominate the field of NLP with their mass adoption in tasks such as text generation, summarization and fake news detection. These models offer ease of deployment and reliability for most applications, however, they require significant amounts of computational power for training as well as inference. This poses challenges in their adoption in resource-constrained applications, especially in the open-source community where compute availability is usually scarce. This work proposes a graph-based approach for Environmental Claim Detection, exploring Graph Neural Networks (GNNs) and Hyperbolic Graph Neural Networks (HGNNs) as lightweight yet effective alternatives to transformer-based models. Re-framing the task as a graph classification problem, we transform claim sentences into dependency parsing graphs, utilizing a combination of word2vec \& learnable part-of-speech (POS) tag embeddings for the node features and encoding syntactic dependencies in the edge relations. Our results show that our graph-based models, particularly HGNNs in the poincaré space (P-HGNNs), achieve performance superior to the state-of-the-art on environmental claim detection while using up to \textbf{30x fewer parameters}. We also demonstrate that HGNNs benefit vastly from explicitly modeling data in hierarchical (tree-like) structures, enabling them to significantly improve over their euclidean counterparts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。