研究生如何评估AI生成内容的权威性，研究揭示了三类判断依据。
Evaluating Machine Expertise: How Graduate Students Develop Frameworks for Assessing GenAI Content
- 基于专业身份、验证能力与系统使用经验构建评估框架
- 不同专业领域对AI输出的接受度差异显著，核心工作保留自主权
- 适合关注人机协作、AI可信度评估的研究者和教育工作者
本研究通过调查问卷、LLM交互记录及14名研究生的深度访谈，探讨研究生在网页交互中如何发展评估大语言模型（LLM）生成内容专业性的框架。研究发现，学生评估行为受三大因素影响：专业身份认知、内容验证能力与系统导航经验。他们并非全盘接受或拒绝输出，而是保护与其专业身份紧密相关的领域——管理者维护概念性工作，设计师守护创意过程，程序员掌控核心技术。评估框架还受到验证不同类型内容的能力以及复杂系统使用经验的影响。该研究为网络科学提供了新兴人-生成式AI交互模式的洞察，并建议平台应更好支持用户在人工智能中介的网络环境中建立有效的机器生成专业知识信号评估机制。
原文摘要 · Abstract (English)
This paper examines how graduate students develop frameworks for evaluating machine-generated expertise in web-based interactions with large language models (LLMs). Through a qualitative study combining surveys, LLM interaction transcripts, and in-depth interviews with 14 graduate students, we identify patterns in how these emerging professionals assess and engage with AI-generated content. Our findings reveal that students construct evaluation frameworks shaped by three main factors: professional identity, verification capabilities, and system navigation experience. Rather than uniformly accepting or rejecting LLM outputs, students protect domains central to their professional identities while delegating others--with managers preserving conceptual work, designers safeguarding creative processes, and programmers maintaining control over core technical expertise. These evaluation frameworks are further influenced by students' ability to verify different types of content and their experience navigating complex systems. This research contributes to web science by highlighting emerging human-genAI interaction patterns and suggesting how platforms might better support users in developing effective frameworks for evaluating machine-generated expertise signals in AI-mediated web environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。