arXiv:2609.07797cs.CL2026-09

将语法关系引入主题模型,发现其在辩论类文本中提升主题质量,但在技术文档中效果有限。

Does Syntax Matter? A Graph-Augmented Variational Topic Model for Computational Social Sciences

  • 构建包含词汇与句法边的异质图,用图注意力网络增强变分自编码器进行主题推断
  • 语法增强显著提升主题多样性,在所有语料中优于无图基线,但对文档对齐贡献不大
  • 适合分析辩论、倡议等具有立场和论辩结构的文本,不适用于纯信息性或行政文本

主题模型广泛用于计算社会科学中识别大规模文本中的潜在主题。传统方法依赖词袋表示和生成模型如LDA,而近期方法如BERTopic则基于密集文档嵌入。本文提出结构化上下文概率主题模型(SCPTM),将句法依存关系融入主题推断。SCPTM将语料表示为由词汇与句法边连接的文档-词异质图,通过图注意力网络在变分自编码器中处理,生成概率性、混合成员的主题分布。我们在四种注册形式与论述结构不同的语料上评估七种主题模型(含四种SCPTM消融版本)。评估指标包括主题一致性(C_V, C_NPMI)、主题多样性、聚类-标签对齐(NMI)及短语级诊断(互补性与情感差距)。结果表明,SCPTM的神经架构在文档-主题对齐上显著优于生成基线,但该优势主要来自变分编码器而非句法信息。句法信息提升主题多样性,图增强变体在所有语料中均优于无图基线;其对描述符质量的提升体现在捕捉论辩文本中的谓词-论元结构与立场表达,而在技术与制度类文本中冗余。情感差距在所有变体中均为正值,但主要由短语分组驱动而非句法过滤。结论是:句法编码有条件有效——在行动导向、论辩性强的文本中受益,但在信息性或行政类文本中引入噪声。

原文摘要 · Abstract (English)

Topic modeling is widely used in computational social sciences to identify latent themes in large text corpora. Traditional approaches rely on Bag-of-Words representations and generative models such as LDA, while recent methods like BERTopic operate on dense document embeddings. This paper introduces the Structural Contextual Probabilistic Topic Model (SCPTM), an architecture that incorporates syntactic dependency relations into topic inference. SCPTM represents a corpus as a heterogeneous graph of documents and words connected by lexical and syntactic edges, processed through a Graph Attention Network within a Variational Autoencoder to produce probabilistic, mixed-membership topic distributions. We evaluate seven topic modeling techniques (including four SCPTM ablations) across four corpora differing in register and discourse structure. Our framework combines coherence (C_V, C_NPMI), topic diversity, clustering-label alignment (NMI), and phrase-level diagnostics (complementarity and valence gap). Results show that SCPTM's neural architecture yields substantial gains in document-topic alignment over generative baselines, but these gains are attributable to the variational encoder rather than to syntax. Syntax contributes to topic diversity, where graph-augmented variants outperform the no-graph baseline across all corpora, and to descriptor quality: dependency paths capture predicate-argument structures and stance in deliberative registers, while proving redundant in technical and institutional corpora. The valence gap is positive across all variants, but driven primarily by phrase grouping rather than syntactic filtering. We conclude that syntactic encoding matters conditionally: it benefits action-oriented, argumentative texts, but introduces noise in informational or administrative registers.

主题模型句法增强计算社会学图神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。