用AI生成真假历史故事,教中小学生识别信息陷阱。
DoYouTrustAI: A Tool to Teach Students About AI Misinformation and Prompt Engineering
- 设计互动工具,通过真假历史摘要对比训练批判性思维。
- 学生需在无外部帮助下判断AI回答真伪,提升辨识能力。
- 适合教育者用于教学,帮助学生理解提示工程的影响力。
大型语言模型(LLM)如ChatGPT在过去五年中迅速发展并广泛使用,改变了用户对传统搜索引擎的依赖。然而,其生成特性带来了将错误信息当作事实呈现的风险。为此,我们开发了一个面向K-12学生的基于网页的应用程序——DoYouTrustAI,旨在通过交互式教学帮助学生识别LLM回应中的误导性信息。该工具利用提示工程生成关于重要历史人物的摘要,内容可能是真实记录,也可能是有意歪曲的历史版本。学生需在不借助外部资源的情况下判断陈述的真实性。研究问题包括:(RQ1) 如何设计工具帮助学生认识误导性信息及生成式AI的欺骗性?(RQ2) 能否使提示工程概念对学生产生可理解性?结果表明,应在用户内化错误信息前进行纠正。工具选择学生熟悉的个体进行测试,减少随机猜测;同时将误导内容与已知事实并列,增强可信度。此外,提供预设提示指令,展示不同提示如何影响输出。这些功能共同构建了一个可控环境,帮助用户理解验证AI输出的重要性,并掌握提示工程的基本原理。
原文摘要 · Abstract (English)
AI, especially Large Language Models (LLMs) like ChatGPT, have rapidly developed and gained widespread adoption in the past five years, shifting user preference from traditional search engines. However, the generative nature of LLMs raises concerns about presenting misinformation as fact. To address this, we developed a web-based application that helps K-12 students enhance critical thinking by identifying misleading information in LLM responses about major historical figures. In this paper, we describe the implementation and design details of the DoYouTrustAI tool, which can be used to provide an interactive lesson which teaches students about the dangers of misinformation and how believable generative AI can make it seem. The DoYouTrustAI tool utilizes prompt engineering to present the user with AI generated summaries about the life of a historical figure. These summaries can be either accurate accounts of that persons life, or an intentionally misleading alteration of their history. The user is tasked with determining the validity of the statement without external resources. Our research questions for this work were:(RQ1) How can we design a tool that teaches students about the dangers of misleading information and of how misinformation can present itself in LLM responses? (RQ2) Can we present prompt engineering as a topic that is easily understandable for students? Our findings highlight the need to correct misleading information before users retain it. Our tool lets users select familiar individuals for testing to reduce random guessing and presents misinformation alongside known facts to maintain believability. It also provides pre-configured prompt instructions to show how different prompts affect AI responses. Together, these features create a controlled environment where users learn the importance of verifying AI responses and understanding prompt engineering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。