让大模型学会提关键问题，揪出论证漏洞。
Critical Questions Generation: Motivation and Challenges
- 基于论证理论设计问题生成模板，引导模型提问。
- 用大模型自动生成问题，验证其有效性与局限性。
- 适合研究可信AI、批判性思维的学者和工程师。
大型语言模型（LLMs）在应对虚假信息方面表现出色，如生成反驳论点。然而，它们仍受制于过时知识和幻觉内容生成的问题。为此，我们提出一项新任务——关键问题生成（Critical Questions Generation, CQs），即对论证性文本生成其隐含的关键问题。根据沃尔顿的论证理论，关键问题旨在揭示论证中缺失的信息盲点。因此，我们不依赖外部知识让模型生成反驳，而是让其主动质疑论证本身。为支持大规模实验，我们探索两种构建参考数据集的方法：(i) 根据沃尔顿的论证理论实例化关键问题模板；(ii) 使用大模型自身作为关键问题生成器。我们的工作建立了有效关键问题的标准，并发现尽管大模型具备合理的生成能力，但在该任务上仍有显著提升空间。
原文摘要 · Abstract (English)
The development of Large Language Models (LLMs) has brought impressive performances on mitigation strategies against misinformation, such as counterargument generation. However, LLMs are still seriously hindered by outdated knowledge and by their tendency to generate hallucinated content. In order to circumvent these issues, we propose a new task, namely, Critical Questions Generation, consisting of processing an argumentative text to generate the critical questions (CQs) raised by it. In argumentation theory CQs are tools designed to lay bare the blind spots of an argument by pointing at the information it could be missing. Thus, instead of trying to deploy LLMs to produce knowledgeable and relevant counterarguments, we use them to question arguments, without requiring any external knowledge. Research on CQs Generation using LLMs requires a reference dataset for large scale experimentation. Thus, in this work we investigate two complementary methods to create such a resource: (i) instantiating CQs templates as defined by Walton's argumentation theory and (ii), using LLMs as CQs generators. By doing so, we contribute with a procedure to establish what is a valid CQ and conclude that, while LLMs are reasonable CQ generators, they still have a wide margin for improvement in this task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。