arXiv:2507.07414cs.CLcs.AI2025-07被引 1

混合CNN与GNN的文本模型,处理长文本更高效。

GNN-CNN: An Efficient Hybrid Model of Convolutional and Graph Neural Networks for Text Representation

  • 用CNN捕获局部特征,通过图结构扩展感受野。
  • 生成的小世界图平均聚类系数0.45,路径长度4-5,结构合理。
  • 无需填充截断,结合LLM信息,适合长文本分类任务。

时间、成本和能效是深度学习中处理长文本的关键考量。当前主流的Transformer模型随输入长度呈二次方复杂度增长,处理长文档效率低下。本文提出一种融合图神经网络(GNN)与卷积神经网络(CNN)的新型架构,结合实时端到端图生成机制,以字符级小批量输入进行处理,无需填充或截断。为提升性能并保持高效率,模型通过高效字典查找引入大语言模型(LLMs)的词嵌入与情感极性等信息。利用CNN捕捉局部上下文模式,通过基于网格的图结构扩展局部感受野,并采用小世界图聚合文档级信息。生成的图具有有意义的语义组织特性,平均聚类系数约0.45,平均最短路径长度在4至5之间。在情感分析与新闻分类等多个文本分类任务上评估,结果表明该模型在保持高效的同时具备竞争力的性能。

原文摘要 · Abstract (English)

Time, cost, and energy efficiency are critical considerations in Deep-Learning (DL), particularly when processing long texts. Transformers, which represent the current state of the art, exhibit quadratic computational complexity relative to input length, making them inefficient for extended documents. This study introduces a novel model architecture that combines Graph Neural Networks (GNNs) and Convolutional Neural Networks (CNNs), integrated with a real-time, end-to-end graph generation mechanism. The model processes compact batches of character-level inputs without requiring padding or truncation. To enhance performance while maintaining high speed and efficiency, the model incorporates information from Large Language Models (LLMs), such as token embeddings and sentiment polarities, through efficient dictionary lookups. It captures local contextual patterns using CNNs, expands local receptive fields via lattice-based graph structures, and employs small-world graphs to aggregate document-level information. The generated graphs exhibit structural properties indicative of meaningful semantic organization, with an average clustering coefficient of approximately 0.45 and an average shortest path length ranging between 4 and 5. The model is evaluated across multiple text classification tasks, including sentiment analysis and news-categorization, and is compared against state-of-the-art models. Experimental results confirm the proposed model's efficiency and competitive performance.

文本表示图神经网络轻量模型长文本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。