用大模型知识增强事件合理性判断，提升模型理解真实世界的能力。
Plausibility Vaccine: Injecting LLM Knowledge for Event Plausibility
- 通过参数高效微调注入大模型隐含知识，构建多任务适配器
- 在两个数据集上显著提升事件合理性预测准确率
- 适合需要常识推理的NLP应用，如对话生成与内容审核
尽管语言建模取得进展，基于共现分布的语义表示方法仍难以区分合理与不合理事件。本文研究如何通过参数高效微调，从大语言模型中提取潜在知识，以改进事件合理性预测。我们训练了12个任务适配器，学习不同的物理属性与关联度量，并对各任务的潜在语义知识进行融合，叠加在预训练AlBERT嵌入之上。我们自动化生成辅助任务数据，实现方法可扩展性，并在两个合理性数据集上微调所学表征。代码已公开于https://github.com/Jacob-Chmura/plausibility-vaccine。
原文摘要 · Abstract (English)
Despite advances in language modelling, distributional methods that build semantic representations from co-occurrences fail to discriminate between plausible and implausible events. In this work, we investigate how plausibility prediction can be improved by injecting latent knowledge prompted from large language models using parameter-efficient fine-tuning. We train 12 task adapters to learn various physical properties and association measures and perform adapter fusion to compose latent semantic knowledge from each task on top of pre-trained AlBERT embeddings. We automate auxiliary task data generation, which enables us to scale our approach and fine-tune our learned representations across two plausibility datasets. Our code is available at https://github.com/Jacob-Chmura/plausibility-vaccine.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。