比较两种模型对短问卷回复的分析效果,助你选对方法。
A Comparative Evaluation of Structural Topic Models and BERTopic for Short, Open-Ended Survey Responses
- 用嵌入+语义增强提升短文本建模,优于传统概率模型。
- BERTopic主题更清晰稳定,但STM更适合推断性分析。
- 适合心理学等社科研究者参考模型搭配使用策略。
在应用心理学中,主题建模正融合概率词袋模型与基于嵌入的新方法。然而,多数评估依赖较长且干净的语料库,对短而开放的问卷回复指导不足。本文比较了结构化主题模型(STM)与基于嵌入的BERTopic在分析开放问卷回复中的表现。我们测试了三种STM条件和五种BERTopic条件,涵盖拼写修正、词干提取、嵌入选择及一种新提出的上下文增强策略,以增强极短回复的语义信息。结果显示,BERTopic始终获得更高主题一致性,其中上下文增强带来最大提升;而高维嵌入单独使用并未改善一致性,反而导致更多数据丢失。定性分析表明,BERTopic生成的主题更可读且稳定,而STM主题往往更宽泛且混杂。但STM在协变量推断分析方面更强,而BERTopic的协变量比较多为描述性。结果表明,两类模型具有互补优势。最后提出在应用社会科学中选择与结合主题建模方法的实用建议。
原文摘要 · Abstract (English)
Topic modeling in applied psychology increasingly spans two methodological traditions: probabilistic bag-of-words models and newer embedding-based approaches. Yet many evaluations of these methods rely on longer and cleaner benchmark corpora, leaving less guidance for short, open-ended survey responses. This paper compares Structural Topic Models (STM), a probabilistic topic model, and BERTopic, an embedding-based model, for analyzing open-ended survey responses. We evaluated three STM conditions and five BERTopic conditions, varying typographical correction, stemming, embedding choice, and contextual augmentation, a strategy we introduced to provide additional semantic context for very short responses. Results indicate that BERTopic consistently produced higher topic coherence than STM, with contextual augmentation yielding the strongest performance gains. In contrast, higher-dimensional embeddings alone did not improve coherence and were associated with greater data loss. Qualitative evaluation showed that BERTopic generated more interpretable and stable topics, while STM topics were often broader and more mixed. However, STM provides stronger support for inferential covariate analysis, whereas BERTopic covariate comparisons are primarily descriptive. These findings suggest that STM and BERTopic offer complementary strengths. We conclude with practical guidance for selecting and combining topic modeling approaches in applied social science research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。