用事件块构建文本语义图,提升无监督文本表示学习效率
SE-GCL: An Event-Based Simple and Effective Graph Contrastive Learning for Text Representation
- 从文本中提取事件块构建内部关系图,捕捉核心语义关联
- 通过简化数据增强和多损失函数,实现高效特征学习
- 在4个新闻数据集上表现优异,适合追求高效语义建模的场景
文本表示学习是自然语言处理的基础。近年来,图对比学习(GCL)因其能在自监督设置下捕捉复杂文本信息而被广泛应用。然而,现有主流GCL方法常需依赖领域知识或复杂的计算来指导数据增强,显著限制了其应用效率与范围。此外,许多方法仅基于词-文档关系建模,忽略了文本中的丰富上下文语义。为此,我们提出一种基于事件的、简单有效的图对比学习(SE-GCL)框架。具体而言,从文本中提取事件块并构建内部关系图,以保留最关键的语义信息;设计轻量级无监督图对比学习框架,利用事件语义与结构信息的互补性捕捉复杂特征。特别地,引入事件骨架概念以表征核心语义,并简化传统GCL中复杂的数据增强技术,提升算法效率。通过多损失函数引导嵌入在向量空间中收敛或发散,实现平衡优化。我们在四个标准数据集(AG News、20NG、SougouNews、THUCNews)上验证了SE-GCL的有效性。
原文摘要 · Abstract (English)
Text representation learning is significant as the cornerstone of natural language processing. In recent years, graph contrastive learning (GCL) has been widely used in text representation learning due to its ability to represent and capture complex text information in a self-supervised setting. However, current mainstream graph contrastive learning methods often require the incorporation of domain knowledge or cumbersome computations to guide the data augmentation process, which significantly limits the application efficiency and scope of GCL. Additionally, many methods learn text representations only by constructing word-document relationships, which overlooks the rich contextual semantic information in the text. To address these issues and exploit representative textual semantics, we present an event-based, simple, and effective graph contrastive learning (SE-GCL) for text representation. Precisely, we extract event blocks from text and construct internal relation graphs to represent inter-semantic interconnections, which can ensure that the most critical semantic information is preserved. Then, we devise a streamlined, unsupervised graph contrastive learning framework to leverage the complementary nature of the event semantic and structural information for intricate feature data capture. In particular, we introduce the concept of an event skeleton for core representation semantics and simplify the typically complex data augmentation techniques found in existing graph contrastive learning to boost algorithmic efficiency. We employ multiple loss functions to prompt diverse embeddings to converge or diverge within a confined distance in the vector space, ultimately achieving a harmonious equilibrium. We conducted experiments on the proposed SE-GCL on four standard data sets (AG News, 20NG, SougouNews, and THUCNews) to verify its effectiveness in text representation learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。