用提示词设计提升AI生成的质性编码质量
Prompts Matter: Comparing ML/GAI Approaches for Generating Inductive Qualitative Coding Results
- 将人类编码流程融入GAI提示词，优化生成结果
- 新方法在在线社区数据上显著优于传统ML/GAI方案
- 适合教育研究者快速实现高质量质性分析
质性研究中的归纳编码方法在教育领域应用已久，但耗时耗力。近年来，生成式人工智能（GAI）在生成归纳编码结果方面初现成效。与人类编码者类似，GAI的表现高度依赖指令设计。本研究采用两种已知方法和两种基于理论的新方法，对一个在线社区数据集进行编码结果生成，并评估其效果。结果显示，不同机器学习/生成式AI方法间存在显著差异，而引入人类编码过程的新提示设计方法表现更优。
原文摘要 · Abstract (English)
Inductive qualitative methods have been a mainstay of education research for decades, yet it takes much time and effort to conduct rigorously. Recent advances in artificial intelligence, particularly with generative AI (GAI), have led to initial success in generating inductive coding results. Like human coders, GAI tools rely on instructions to work, and how to instruct it may matter. To understand how ML/GAI approaches could contribute to qualitative coding processes, this study applied two known and two theory-informed novel approaches to an online community dataset and evaluated the resulting coding results. Our findings show significant discrepancies between ML/GAI approaches and demonstrate the advantage of our approaches, which introduce human coding processes into GAI prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。