比较大学生对AI、人工及人机协作反馈的信任度，发现透明度影响信任但需提升反馈素养。
Evaluating Trust in AI, Human, and Co-produced Feedback Among Undergraduate Students
- 通过91名学生实验，对比三类反馈源的可信度差异。
- 盲源时学生更信任AI与人机协作反馈，但公开来源后偏见显著上升。
- 提升AI使用经验可增强识别能力，但长期使用反而降低信任感。
随着生成式AI模型（尤其是大语言模型，LLMs）在高等教育（HE）中改变教学反馈实践，理解学生对不同反馈来源的感知变得至关重要。本研究通过一项包含91名参与者的被试内实验设计，探讨了学生区分反馈类型的能力、对反馈质量的评价以及来源相关的潜在偏见。结果显示，当反馈来源匿名时，学生普遍认为AI和人机协作反馈在实用性与客观性上优于人工反馈；但当来源披露后，学生对AI反馈表现出强烈偏见。此外，仅在来源公开时，AI反馈的真诚感感知下降，而人机协作反馈仍保持积极评价。拥有更多教育类AI使用经验的学生，更擅长识别LLM生成的反馈，并对各类反馈信任度更高；然而，使用一般AI时间越长的学生，对反馈的实用性与可信度感知反而越低。这些发现揭示了来源可信度的重要性，强调需提升反馈素养与AI素养，以减少对学生对AI反馈感知的偏见，推动其在教育中的有效采纳与影响。
原文摘要 · Abstract (English)
As generative AI models, particularly large language models (LLMs), transform educational feedback practices in higher education (HE) contexts, understanding students' perceptions of different sources of feedback becomes crucial for their effective implementation and adoption. This study addresses a critical gap by comparing undergraduate students' trust in LLM, human, and human-AI co-produced feedback in their authentic HE context. More specifically, through a within-subject experimental design involving 91 participants, we investigated factors that predict students' ability to distinguish between feedback types, their perceptions of feedback quality, and potential biases related to the source of feedback. Findings revealed that when the source was blinded, students generally preferred AI and co-produced feedback over human feedback regarding perceived usefulness and objectivity. However, they presented a strong bias against AI when the source of feedback was disclosed. In addition, only AI feedback suffered a decline in perceived genuineness when feedback sources were revealed, while co-produced feedback maintained its positive perception. Educational AI experience improved students' ability to identify LLM-generated feedback and increased their trust in all types of feedback. More years of students' experience using AI for general purposes were associated with lower perceived usefulness and credibility of feedback. These insights offer substantial evidence of the importance of source credibility and the need to enhance both feedback literacy and AI literacy to mitigate bias in student perceptions for AI-generated feedback to be adopted and impact education.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。