用自定义GPT辅助编程课堂对话编码,提升分析效率。
Exploring Effective Strategies for Building a User-Configured GPT for Coding Classroom Dialogues
- 基于GPT-4构建定制化对话分析工具MyGPT,适配小数据集
- 在人工编码体系下,模型生成建议准确率可观,可作辅助
- 提出可复现的配置策略,适合做教育对话研究的研究者
本研究探索了构建用于编程课堂对话分析的自定义GPT的有效策略。尽管课堂对话是教育中的关键环节,但其分析因需深入理解对话功能且依赖人工转录编码而困难重重。大语言模型(LLMs)为自动化此过程提供了可能,但现有研究多聚焦于大规模模型训练或固定编码体系下的预训练模型评估,结果难适用,方法也难以复现。本文以基于GPT-4的MyGPT系统为例,评估其在使用人工编码体系时对课堂对话的基准表现,并在受控变量设计下,考察不同示例输入对性能的影响。通过设计型研究方法,探索了一套基于MyGPT特性的实用策略,用于在有限数据条件下构建有效工具。研究发现,尽管存在一些局限,但采用这些策略构建的自定义GPT可作为可靠的编码助手,生成有价值的编码建议。
原文摘要 · Abstract (English)
This study investigated effective strategies for developing a custom GPT to code classroom dialogue. While classroom dialogue is widely recognised as a crucial element of education, its analysis remains challenging due to the need for a nuanced understanding of dialogic functions and the labour-intensive nature of manual transcript coding. Recent advancements in large language models (LLMs) offer promising avenues for automating this process. However, existing studies predominantly focus on training large-scale models or evaluating pre-trained models with fixed codebooks, the outcomes of which are often not applicable, or the methods are not replicable for dialogue researchers working with small datasets or employing customised coding schemes. Using MyGPT - a GPT-4-based customised GPT system configured for dialogue analysis - as a case, this study evaluates its baseline performance in coding classroom dialogue with a human codebook and examines how performance varies with different example inputs under a controlled variable design. Through a design-based research approach, this study explores a set of practical strategies - based upon MyGPT's unique features - for configuring an effective tool with limited data. The findings suggest that, despite a few limitations, a custom GPT developed using these specific strategies can serve as a useful coding assistant by generating coding suggestions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。