arXiv:2410.09696cs.LGstat.ML2024-10被引 1

用威布尔分布改进图注意力模型,更好捕捉文档网络的层级关系。

Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks

  • 引入图泊松因子分析与伽马信念网络,实现多层语义关系建模。
  • 在多个数据集上生成高质量层次化文档表示,图任务性能显著提升。
  • 适合研究文档网络、主题建模与图神经网络融合的学者使用。

尽管变分图自编码器(VGAE)被广泛用于建模和生成图结构数据,但大多数方法仍难以准确逼近稀疏且偏斜的潜在节点表示,尤其在具有离散观测值的文档关系网络(DRNs)中表现不足。为分析互联文档集合,贝叶斯模型中的关系主题模型(RTMs)已被证明能有效描述DRNs的链接结构与文档内容,这促使我们尝试将RTMs与现有VGAE结合,以缓解其在建模DRN生成过程中的局限性。本文突破传统RTMs复杂的近似假设,提出图泊松因子分析(GPFA),提供解析的条件后验以提升推断精度,并将其扩展为多随机层版本——图泊松伽马信念网络(GPGBN),用于捕获多语义层级的文档关系。以GPGBN作为解码器,结合多种基于威布尔分布的图推理网络,构建两种威布尔图自编码器(WGAE)变体,并配备相应的模型推断算法。实验表明,所提模型能提取高质量的层次化潜在文档表示,在多种图分析任务中表现优异。

原文摘要 · Abstract (English)

Although existing variational graph autoencoders (VGAEs) have been widely used for modeling and generating graph-structured data, most of them are still not flexible enough to approximate the sparse and skewed latent node representations, especially those of document relational networks (DRNs) with discrete observations. To analyze a collection of interconnected documents, a typical branch of Bayesian models, specifically relational topic models (RTMs), has proven their efficacy in describing both link structures and document contents of DRNs, which motives us to incorporate RTMs with existing VGAEs to alleviate their potential issues when modeling the generation of DRNs. In this paper, moving beyond the sophisticated approximate assumptions of traditional RTMs, we develop a graph Poisson factor analysis (GPFA), which provides analytic conditional posteriors to improve the inference accuracy, and extend GPFA to a multi-stochastic-layer version named graph Poisson gamma belief network (GPGBN) to capture the hierarchical document relationships at multiple semantic levels. Then, taking GPGBN as the decoder, we combine it with various Weibull-based graph inference networks, resulting in two variants of Weibull graph auto-encoder (WGAE), equipped with model inference algorithms. Experimental results demonstrate that our models can extract high-quality hierarchical latent document representations and achieve promising performance on various graph analytic tasks.

图神经网络文档建模主题模型层级表示

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。