用提示调优提升大模型判断信息是否需核查的准确率。
Application and Optimization of Large Models Based on Prompt Tuning for Fact-Check-Worthiness Estimation
- 用设计好的提示模板让大模型在上下文中学习判断标准。
- 在公开数据集上F1和准确率优于或匹配BERT、GPT-3.5等模型。
- 特别适合数据少或无标签时，快速构建可信度评估系统。
针对全球化与信息化背景下虚假信息泛滥的问题，本文提出一种基于提示调优的事实核查必要性分类方法。通过在大语言模型中应用定制的提示模板，实现上下文学习，并利用提示调优技术提升判断陈述是否需核查的准确性，尤其在数据有限或无标注的情况下表现优异。在多个公开数据集上的大量实验表明，该方法在事实核查必要性分类任务中，性能优于或持平于包括BERT在内的经典预训练模型，以及GPT-3.5、GPT-4等最新大模型。实验结果显示，该方法在F1分数和准确率等评价指标上具有明显优势，充分验证了其在该任务中的有效性与先进性。
原文摘要 · Abstract (English)
In response to the growing problem of misinformation in the context of globalization and informatization, this paper proposes a classification method for fact-check-worthiness estimation based on prompt tuning. We construct a model for fact-check-worthiness estimation at the methodological level using prompt tuning. By applying designed prompt templates to large language models, we establish in-context learning and leverage prompt tuning technology to improve the accuracy of determining whether claims have fact-check-worthiness, particularly when dealing with limited or unlabeled data. Through extensive experiments on public datasets, we demonstrate that the proposed method surpasses or matches multiple baseline methods in the classification task of fact-check-worthiness estimation assessment, including classical pre-trained models such as BERT, as well as recent popular large models like GPT-3.5 and GPT-4. Experiments show that the prompt tuning-based method proposed in this study exhibits certain advantages in evaluation metrics such as F1 score and accuracy, thereby effectively validating its effectiveness and advancement in the task of fact-check-worthiness estimation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。