arXiv:2512.00204cs.CLcs.AI2025-12

用依存树匹配提升自然语言推理效率,参数少、速度快、效果好。

Tree Matching Networks for Natural Language Inference: Parameter-Efficient Semantic Understanding via Dependency Parse Trees

  • 基于依存句法树设计图匹配网络,利用语法结构先验知识
  • 在SNLI任务上性能超越BERT,参数量减少、训练快3倍以上
  • 适合资源受限场景下的语义理解,如移动端或嵌入式系统

在自然语言推理(NLI)任务中,基于Transformer的模型如BERT虽精度高,但需数亿参数,计算开销大。这类模型将句子视为词元序列,从零学习词间关系。而若直接输入依赖句法树等显式语言结构,可利用已有语法关系信息,避免重复学习,提升效率。为此,本文将图匹配网络(GMN)适配为树匹配网络(TMN),在SNLI蕴含任务和SemEval相似性任务上与BERT对比。TMN在SNLI任务上显著优于BERT,内存占用更低,训练时间减少超3倍;但在SemEval任务上表现不佳。显式结构表示在相当规模下显著优于序列模型,但现有聚合方法限制可扩展性。本文提出多头注意力聚合机制以解决该问题。

原文摘要 · Abstract (English)

In creating sentence embeddings for Natural Language Inference (NLI) tasks, using transformer-based models like BERT leads to high accuracy, but require hundreds of millions of parameters. These models take in sentences as a sequence of tokens, and learn to encode the meaning of the sequence into embeddings such that those embeddings can be used reliably for NLI tasks. Essentially, every word is considered against every other word in the sequence, and the transformer model is able to determine the relationships between them, entirely from scratch. However, a model that accepts explicit linguistic structures like dependency parse trees may be able to leverage prior encoded information about these relationships, without having to learn them from scratch, thus improving learning efficiency. To investigate this, we adapt Graph Matching Networks (GMN) to operate on dependency parse trees, creating Tree Matching Networks (TMN). We compare TMN to a BERT based model on the SNLI entailment task and on the SemEval similarity task. TMN is able to achieve significantly better results with a significantly reduced memory footprint and much less training time than the BERT based model on the SNLI task, while both models struggled to preform well on the SemEval. Explicit structural representations significantly outperform sequence-based models at comparable scales, but current aggregation methods limit scalability. We propose multi-headed attention aggregation to address this limitation.

自然语言推理依存句法高效模型图神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。