用大模型把政治辩论文本自动提炼成层级化主题,提升分析效率。
From Quotes to Concepts: Axial Coding of Political Debates with Ensemble LMs
- 用多模型协作+大模型协调,将原始发言归纳为高阶主题类别。
- 聚类方法覆盖更全(高覆盖率),直接大模型分组更精细但漏掉20%内容。
- 结果结构清晰可解释,适合研究政治话语、社会议题的学者使用。
轴向编码是一种常用的定性分析方法,通过将句子级开放编码组织为更广泛的主题类别来增强文档理解。本文将大语言模型(LLMs)应用于轴向编码过程。在基于集成模型的开放编码基础上,引入一个大模型作为协调者,增加轴向编码步骤,将开放编码归类到更高层级的主题中,从而将冗长的议会辩论转录本转化为简洁、层次化的表达。我们对比了两种策略:(i) 使用密度和划分算法对编码-语句对嵌入进行聚类,再由大模型标注类别;(ii) 直接通过大模型将编码与语句分组到主题。方法应用于荷兰议会辩论数据集,生成紧凑且层级分明的编码与主题结构。评估采用外在指标(与人工标注主题一致性的ROUGE-L、余弦相似度、BERTScore)和内在指标(覆盖度、简洁性、连贯性、新颖性、JSD散度)。结果表明:密度聚类具有更高的覆盖度和集群对齐性,而直接大模型分组虽更精细,但覆盖度低20%。总体上,聚类方法在覆盖范围和结构分离上表现更优,而大模型分组生成的类别更简洁、可读性强且语义对齐。为支持后续研究,我们公开发布完整语句与编码数据集,保障可复现性与比较研究。
原文摘要 · Abstract (English)
Axial coding is a commonly used qualitative analysis method that enhances document understanding by organizing sentence-level open codes into broader categories. In this paper, we operationalize axial coding with large language models (LLMs). Extending an ensemble-based open coding approach with an LLM moderator, we add an axial coding step that groups open codes into higher-order categories, transforming raw debate transcripts into concise, hierarchical representations. We compare two strategies: (i) clustering embeddings of code-utterance pairs using density-based and partitioning algorithms followed by LLM labeling, and (ii) direct LLM-based grouping of codes and utterances into categories. We apply our method to Dutch parliamentary debates, converting lengthy transcripts into compact, hierarchically structured codes and categories. We evaluate our method using extrinsic metrics aligned with human-assigned topic labels (ROUGE-L, cosine, BERTScore), and intrinsic metrics describing code groups (coverage, brevity, coherence, novelty, JSD divergence). Our results reveal a trade-off: density-based clustering achieves high coverage and strong cluster alignment, while direct LLM grouping results in higher fine-grained alignment, but lower coverage 20%. Overall, clustering maximizes coverage and structural separation, whereas LLM grouping produces more concise, interpretable, and semantically aligned categories. To support future research, we publicly release the full dataset of utterances and codes, enabling reproducibility and comparative studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。