arXiv:2510.24337cs.AIcs.SI2025-10被引 1

用大模型做传播研究内容分析,效率高且省成本。

Generative Large Language Models (gLLMs) in Content Analysis: A Practical Guide for Communication Research

  • 用自然语言指令让大模型完成编码任务,无需复杂编程。
  • 比人工编码更快更便宜,还能理解隐含语境和深层含义。
  • 提供完整操作指南,助研究者规避七类关键方法风险。

生成式大语言模型(gLLMs)如ChatGPT正被越来越多地应用于传播学内容分析。研究表明,gLLMs在多种编码任务上表现优于众包人员和训练有素的研究助理,且耗时和成本仅为后者的几分之一。它们能解析隐含意义与上下文信息,通过自然语言指令操作,仅需基础编程技能,且除验证数据集外几乎无需标注数据,标志着自动化内容分析的范式变革。然而,其在传播研究中的方法论整合仍不充分。在基于gLLMs的定量内容分析中,研究者需应对至少七个关键挑战:(1)代码本设计,(2)提示工程,(3)模型选择,(4)参数调优,(5)迭代优化,(6)模型可靠性验证,以及可选的(7)性能提升。本文综述了相关前沿研究,提出一套全面的最佳实践指南,旨在使gLLM辅助的内容分析更易为传播学研究者掌握,并确保研究在效度、信度、可复现性和伦理方面符合学科标准。

原文摘要 · Abstract (English)

Generative Large Language Models (gLLMs), such as ChatGPT, are increasingly being used in communication research for content analysis. Studies show that gLLMs can outperform both crowd workers and trained coders, such as research assistants, on various coding tasks relevant to communication science, often at a fraction of the time and cost. Additionally, gLLMs can decode implicit meanings and contextual information, be instructed using natural language, deployed with only basic programming skills, and require little to no annotated data beyond a validation dataset - constituting a paradigm shift in automated content analysis. Despite their potential, the integration of gLLMs into the methodological toolkit of communication research remains underdeveloped. In gLLM-assisted quantitative content analysis, researchers must address at least seven critical challenges that impact result quality: (1) codebook development, (2) prompt engineering, (3) model selection, (4) parameter tuning, (5) iterative refinement, (6) validation of the model's reliability, and optionally, (7) performance enhancement. This paper synthesizes emerging research on gLLM-assisted quantitative content analysis and proposes a comprehensive best-practice guide to navigate these challenges. Our goal is to make gLLM-based content analysis more accessible to a broader range of communication researchers and ensure adherence to established disciplinary quality standards of validity, reliability, reproducibility, and research ethics.

大模型内容分析传播研究方法指南

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。