用大模型自动生成提示，实现更高效的自动证据匹配。
Agent-based Automated Claim Matching with Instruction-following LLMs
- 分两步走：先由大模型生成提示，再用大模型做二分类匹配。
- 自动生成的提示比人工提示效果更好，小模型也能达到大模型水平。
- 不同步骤用不同模型，可提升效率，适合自动化系统开发者。
我们提出一种基于代理的自动化声明匹配方法，利用指令遵循型大语言模型（LLMs）。采用两阶段流程：首先由大模型生成提示，然后将声明匹配作为二分类任务进行。实验表明，大模型生成的提示优于人类设计的提示，且在生成阶段使用小型大模型即可达到大型模型的效果，从而节省计算资源。此外，使用不同大模型分别处理提示生成与匹配任务也表现出良好效果。对提示生成过程的分析进一步揭示了大模型对声明匹配任务的理解能力。
原文摘要 · Abstract (English)
We present a novel agent-based approach for the automated claim matching task with instruction-following LLMs. We propose a two-step pipeline that first generates prompts with LLMs, to then perform claim matching as a binary classification task with LLMs. We demonstrate that LLM-generated prompts can outperform SOTA with human-generated prompts, and that smaller LLMs can do as well as larger ones in the generation process, allowing to save computational resources. We also demonstrate the effectiveness of using different LLMs for each step of the pipeline, i.e. using an LLM for prompt generation, and another for claim matching. Our investigation into the prompt generation process in turn reveals insights into the LLMs' understanding of claim matching.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。