arXiv:2511.18843cs.CLcs.HC2025-11被引 1

用BERTopic分析小规模焦点小组数据,提升主题建模可复现性与可解释性

A Reproducible Framework for Neural Topic Modeling in Focus Group Analysis

  • 系统化配置BERTopic,结合多重评估指标优化主题模型
  • 7主题模型比LDA高18%主题一致性,人工验证一致率达0.7以上
  • 揭示稳定性与一致性指标冲突,强调多维度评估必要性

焦点小组讨论生成丰富的定性数据,但传统人工编码效率低且难以复现。本文提出一个系统性框架,将BERTopic应用于突尼斯10个焦点小组(共1,075条发言)关于宫颈癌疫苗认知的访谈文本。在27种超参数配置下进行综合评估,通过自助法稳定性分析、性能指标对比及与LDA基线比较,发现稳定性指标(NMI和ARI)间存在强负相关(r = -0.691),且与主题一致性关系不一致,表明稳定性是多维而非单一概念。基于多准则选择的7主题模型较优化后LDA主题一致性提升18%(0.573 vs. 0.486),并通过独立人工评估验证了主题可解释性(ICC = 0.700,加权Cohen's kappa = 0.678)。结果表明,经系统配置与验证,基于Transformer的主题模型可在小规模焦点组数据中提取可解释主题,同时揭示质量指标反映不同甚至矛盾的构建,需多维度评估。本文提供完整文档与代码以保障可复现性。

原文摘要 · Abstract (English)

Focus group discussions generate rich qualitative data but their analysis traditionally relies on labor-intensive manual coding that limits scalability and reproducibility. We present a systematic framework for applying BERTopic to focus group transcripts using data from ten focus groups exploring HPV vaccine perceptions in Tunisia (1,075 utterances). We conducted comprehensive hyperparameter exploration across 27 configurations, evaluating each through bootstrap stability analysis, performance metrics, and comparison with LDA baseline. Bootstrap analysis revealed that stability metrics (NMI and ARI) exhibited strong disagreement (r = -0.691) and showed divergent relationships with coherence, demonstrating that stability is multifaceted rather than monolithic. Our multi-criteria selection framework yielded a 7-topic model achieving 18\% higher coherence than optimized LDA (0.573 vs. 0.486) with interpretable topics validated through independent human evaluation (ICC = 0.700, weighted Cohen's kappa = 0.678). These findings demonstrate that transformer-based topic modeling can extract interpretable themes from small focus group transcript corpora when systematically configured and validated, while revealing that quality metrics capture distinct, sometimes conflicting constructs requiring multi-criteria evaluation. We provide complete documentation and code to support reproducibility.

主题建模聚焦小组BERTopic可复现性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。