用两阶段提示让大模型先查知识再答题，提升常识推理准确率。
Language Models Benefit from Preparation with Elicited Knowledge
- 第一模型生成相关知识，第二模型基于知识答题
- 在多个数据集上平均准确率超越现有方法
- 无需领域定制提示，适用于多种常识推理任务
零样本链式思维（CoT）常用于需要多步推理的问答任务，但部分任务更依赖知识获取而非推理链。本文提出一种简单提示技巧PREP：使用两个语言模型（LM1和LM2），LM1生成相关信息，LM2接收这些信息后回答问题。该设计旨在更好利用模型的指令遵循能力。PREP基于包含100个问答的问题集，源自一个详尽的图示数据集，涉及物品部件与材料组成关系，问题聚焦于两个物品中哪个更不可能共享材料。我们在该部件-材料数据集及三个公开常识推理数据集上测试方法，结果显示其平均准确率在所有数据集上均高于其他对比方法。
原文摘要 · Abstract (English)
The zero-shot chain of thought (CoT) approach is often used in question answering (QA) by language models (LMs) for tasks that require multiple reasoning steps. However, some QA tasks hinge more on accessing relevant knowledge than on chaining reasoning steps. We introduce a simple prompting technique, called PREP, that involves using two instances of LMs: the first (LM1) generates relevant information, and the second (LM2) receives the information from the user and answers the question. This design is intended to make better use of the LM's instruction-following capability. PREP is applicable across various QA tasks without domain-specific prompt engineering. PREP is developed on a dataset of 100 QA questions, derived from an extensive schematic dataset specifying artifact parts and material composition. These questions ask which of two artifacts is less likely to share materials with another artifact. Such questions probe the LM's knowledge of shared materials in the part structure of different artifacts. We test our method on our parts-and-materials dataset and three published commonsense reasoning datasets. The average accuracy of our method is consistently higher than that of all the other tested methods across all the tested datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。