用一阶逻辑翻译提升大模型对句子语义结构的理解能力
Learning Semantic Structure through First-Order-Logic Translation
- 将句子转化为一阶逻辑形式，让模型更准确识别谓词与论元关系
- 微调大模型在合成数据上进行一阶逻辑翻译，效果优于问答任务
- 适合研究语言模型语义理解、知识推理的科研人员
本文探讨基于Transformer的语言模型能否从简单句子中提取谓词-论元结构。我们发现，语言模型常混淆谓词与对象的对应关系。为缓解此问题，研究了两种任务：问答（Q/A）和一阶逻辑（FOL）翻译，以及提示（prompting）与微调（finetuning）两种训练方式。在FOL翻译任务中，我们在设计的合成数据集上微调多个大语言模型，以评估其泛化能力；在问答任务中，对BERT、RoBERTa等编码器模型进行微调，并对大模型使用提示方法。结果表明，大模型在进行一阶逻辑翻译时，更擅长学习谓词-论元结构。
原文摘要 · Abstract (English)
In this paper, we study whether transformer-based language models can extract predicate argument structure from simple sentences. We firstly show that language models sometimes confuse which predicates apply to which objects. To mitigate this, we explore two tasks: question answering (Q/A), and first order logic (FOL) translation, and two regimes, prompting and finetuning. In FOL translation, we finetune several large language models on synthetic datasets designed to gauge their generalization abilities. For Q/A, we finetune encoder models like BERT and RoBERTa and use prompting for LLMs. The results show that FOL translation for LLMs is better suited to learn predicate argument structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。