arXiv:2603.08406cs.HCcs.CL2026-03被引 1

用AI自动化教育对话分析,提升效率与可信度。

Sandpiper: Orchestrated AI-Annotation for Educational Discourse at Scale

  • 人机协同系统结合交互界面与大模型,实现高效分析。
  • 自动去标识化保障隐私,代码本约束防止幻觉。
  • 支持持续评估,适合教育研究者快速开展质性分析。

数字教育环境正朝着复杂的人机对话发展,为学习与教学过程的研究提供了海量数据。然而,传统质性分析仍存在劳动密集的瓶颈,严重制约研究规模。本文提出Sandpiper,一种混合主动式系统,旨在连接大规模对话数据与人类质性分析能力。通过将交互式研究仪表盘与代理型大语言模型(LLM)引擎紧密集成,平台在不牺牲方法严谨性的前提下实现可扩展分析。Sandpiper通过上下文感知的自动化去标识化流程,并依托安全的高校托管基础设施,解决教育领域中AI应用的关键障碍。系统采用结构化约束编排机制,有效消除LLM幻觉,并严格遵循质性编码手册。内置评估引擎支持持续对比AI与人工标注表现,推动模型迭代优化与验证。我们设计用户研究,评估该系统在提升研究效率、增强评分者一致性及研究人员对AI辅助工作流信任度方面的有效性。

原文摘要 · Abstract (English)

Digital educational environments are expanding toward complex AI and human discourse, providing researchers with an abundance of data that offers deep insights into learning and instructional processes. However, traditional qualitative analysis remains a labor-intensive bottleneck, severely limiting the scale at which this research can be conducted. We present Sandpiper, a mixed-initiative system designed to serve as a bridge between high-volume conversational data and human qualitative expertise. By tightly coupling interactive researcher dashboards with agentic Large Language Model (LLM) engines, the platform enables scalable analysis without sacrificing methodological rigor. Sandpiper addresses critical barriers to AI adoption in education by implementing context-aware, automated de-identification workflows supported by secure, university-housed infrastructure to ensure data privacy. Furthermore, the system employs schema-constrained orchestration to eliminate LLM hallucinations and enforces strict adherence to qualitative codebooks. An integrated evaluations engine allows for the continuous benchmarking of AI performance against human labels, fostering an iterative approach to model refinement and validation. We propose a user study to evaluate the system's efficacy in improving research efficiency, inter-rater reliability, and researcher trust in AI-assisted qualitative workflows.

AI标注教育研究大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。