通过即时反馈,人能学会分辨AI与人类写作文本。
Humans can learn to detect AI-generated texts, or at least learn when they can't
- 提供即时反馈后,参与者辨别准确率显著提升。
- 无反馈组在自信时错误最多,反馈组则明显改善判断。
- 适合教育场景中训练学生识别AI生成内容。
本研究探讨了个体在获得即时反馈的情况下,能否准确区分人类写作与AI生成文本,并利用反馈调整自我能力认知。实验使用GPT-4o生成数百篇涵盖多种文体的文本,与Koditex语料库中的人类写作文本相匹配。254名捷克语母语者参与实验,随机分为两组:一组每轮测试后获得即时反馈,另一组仅在实验结束后获反馈。记录了识别准确率、信心水平、反应时间、文本可读性判断及人口统计学信息和先前对AI技术的接触情况。结果显示,接受即时反馈的参与者在准确性和信心校准上均有显著提升。初始时参与者对AI文本特征存在误解,如认为其风格僵硬、可读性差。值得注意的是,未获反馈组在最自信时犯错最多,而反馈组该问题基本消除。表明通过有针对性的带反馈训练,人们可有效习得区分能力,纠正对AI文本风格和可读性的错误认知,同时促进更准确的自我评估。这一发现对教育场景具有重要意义。
原文摘要 · Abstract (English)
This study investigates whether individuals can learn to accurately discriminate between human-written and AI-produced texts when provided with immediate feedback, and if they can use this feedback to recalibrate their self-perceived competence. We also explore the specific criteria individuals rely upon when making these decisions, focusing on textual style and perceived readability. We used GPT-4o to generate several hundred texts across various genres and text types comparable to Koditex, a multi-register corpus of human-written texts. We then presented randomized text pairs to 254 Czech native speakers who identified which text was human-written and which was AI-generated. Participants were randomly assigned to two conditions: one receiving immediate feedback after each trial, the other receiving no feedback until experiment completion. We recorded accuracy in identification, confidence levels, response times, and judgments about text readability along with demographic data and participants' engagement with AI technologies prior to the experiment. Participants receiving immediate feedback showed significant improvement in accuracy and confidence calibration. Participants initially held incorrect assumptions about AI-generated text features, including expectations about stylistic rigidity and readability. Notably, without feedback, participants made the most errors precisely when feeling most confident -- an issue largely resolved among the feedback group. The ability to differentiate between human and AI-generated texts can be effectively learned through targeted training with explicit feedback, which helps correct misconceptions about AI stylistic features and readability, as well as potential other variables that were not explored, while facilitating more accurate self-assessment. This finding might be particularly important in educational contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。