arXiv:2410.03721cs.CLcs.AI2024-10被引 23

用开源AI自动生成社科研究编码本,省去人工编码烦恼。

Thematic Analysis with Open-Source Generative AI and Machine Learning: A New Method for Inductive Qualitative Codebook Development

  • 用开源大模型+提示工程自动归纳文本主题,模拟人工编码过程。
  • 三组模拟数据测试显示,生成的编码本与原始主题高度匹配。
  • 适合想快速构建编码体系的社科研究者,尤其擅长处理大量文本。

本文探讨开源生成式文本模型在社会科学质性分析中应用的潜力。提出GATOS工作流,结合开源机器学习、自然语言处理工具与生成模型,实现从文本中归纳主题编码本。通过三个模拟数据集验证方法有效性:团队反馈、组织伦理文化、后疫情复工态度。研究采用检索增强生成与提示工程,模仿研究人员逐条阅读、判断是否新增编码的传统归纳流程。结果表明,该流程能有效识别出原始数据中的主题与子主题,生成的编码本与真实主题空间高度一致。

原文摘要 · Abstract (English)

This paper aims to answer one central question: to what extent can open-source generative text models be used in a workflow to approximate thematic analysis in social science research? To answer this question, we present the Generative AI-enabled Theme Organization and Structuring (GATOS) workflow, which uses open-source machine learning techniques, natural language processing tools, and generative text models to facilitate thematic analysis. To establish validity of the method, we present three case studies applying the GATOS workflow, leveraging these models and techniques to inductively create codebooks similar to traditional procedures using thematic analysis. Specifically, we investigate the extent to which a workflow comprising open-source models and tools can inductively produce codebooks that approach the known space of themes and sub-themes. To address the challenge of gleaning insights from these texts, we combine open-source generative text models, retrieval-augmented generation, and prompt engineering to identify codes and themes in large volumes of text, i.e., generate a qualitative codebook. The process mimics an inductive coding process that researchers might use in traditional thematic analysis by reading text one unit of analysis at a time, considering existing codes already in the codebook, and then deciding whether or not to generate a new code based on whether the extant codebook provides adequate thematic coverage. We demonstrate this workflow using three synthetic datasets from hypothetical organizational research settings: a study of teammate feedback in teamwork settings, a study of organizational cultures of ethical behavior, and a study of employee perspectives about returning to their offices after the pandemic. We show that the GATOS workflow is able to identify themes in the text that were used to generate the original synthetic datasets.

质性分析生成式AI编码本开源模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。