用代码透明化话语分析，结合词法与语义方法提升可复现性。
Beyond the Black Box: Integrating Lexical and Semantic Methods in Quantitative Discourse Analysis with BERTopic
- 自建Python流程，融合NLTK、spaCy与Sentence Transformers控制预处理
- 通过参数调优实现高连贯性主题（BERTopic+UMAP+HDBSCAN）
- 适合注重方法透明度的社科研究者，尤其历史政治话语分析
定量话语分析因大语言模型和计算工具的兴起而广泛应用，但依赖MAXQDA、NVivo等黑箱软件会削弱方法透明性与研究目标的一致性。本文提出一种混合式、透明的QDA框架，结合词法与语义方法，实现三角验证、可复现性与可解释性。基于历史政治话语案例研究，我们构建了使用NLTK、spaCy和Sentence Transformers的定制化Python流水线，实现对预处理、词形还原和嵌入生成的细粒度控制。进一步详述迭代式BERTopic建模过程，包含UMAP降维、HDBSCAN聚类及c-TF-IDF关键词提取，通过参数调优与多次运行优化主题连贯性与覆盖率。通过将精确词法检索与上下文感知语义聚类对比，主张多层方法以弥补单一方法局限。工作流强调代码级透明性、研究者自主权与方法三角验证在计算话语研究中的重要性。代码与补充材料可在GitHub获取。
原文摘要 · Abstract (English)
Quantitative Discourse Analysis has seen growing adoption with the rise of Large Language Models and computational tools. However, reliance on black box software such as MAXQDA and NVivo risks undermining methodological transparency and alignment with research goals. This paper presents a hybrid, transparent framework for QDA that combines lexical and semantic methods to enable triangulation, reproducibility, and interpretability. Drawing from a case study in historical political discourse, we demonstrate how custom Python pipelines using NLTK, spaCy, and Sentence Transformers allow fine-grained control over preprocessing, lemmatisation, and embedding generation. We further detail our iterative BERTopic modelling process, incorporating UMAP dimensionality reduction, HDBSCAN clustering, and c-TF-IDF keyword extraction, optimised through parameter tuning and multiple runs to enhance topic coherence and coverage. By juxtaposing precise lexical searches with context-aware semantic clustering, we argue for a multi-layered approach that mitigates the limitations of either method in isolation. Our workflow underscores the importance of code-level transparency, researcher agency, and methodological triangulation in computational discourse studies. Code and supplementary materials are available via GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。