用图树结构融合双向信息，提升长文档分类效果
Graph-tree Fusion Model with Bidirectional Information Propagation for Long Document Classification
- 构建图-树架构，分别捕捉句内语法与文档全局关系
- 双向传播信息，实现词→句→文档的多层次上下文增强
- 突破长度限制，适合超长文本分类任务
长文档分类面临内容冗长、结构复杂带来的局部与全局依赖建模难题。现有方法常受限于词元数量且难以充分建模文档内的层级关系。为此，我们提出一种新型模型，结合语法树进行句子编码和文档图进行文档编码，分别捕获细粒度语法关系与更广泛的文档上下文。采用树注意力机制生成句子表示，同时利用图注意力网络建模句间及句内依赖。训练过程中引入从词到句再到文档，以及反向的双向信息传播机制，增强上下文表征能力。该方法实现了对多层级内容的全面理解，有效处理任意长度文本且无词元数量限制。实验表明，该方法在各类长文档分类任务中均表现出色。
原文摘要 · Abstract (English)
Long document classification presents challenges in capturing both local and global dependencies due to their extensive content and complex structure. Existing methods often struggle with token limits and fail to adequately model hierarchical relationships within documents. To address these constraints, we propose a novel model leveraging a graph-tree structure. Our approach integrates syntax trees for sentence encodings and document graphs for document encodings, which capture fine-grained syntactic relationships and broader document contexts, respectively. We use Tree Transformers to generate sentence encodings, while a graph attention network models inter- and intra-sentence dependencies. During training, we implement bidirectional information propagation from word-to-sentence-to-document and vice versa, which enriches the contextual representation. Our proposed method enables a comprehensive understanding of content at all hierarchical levels and effectively handles arbitrarily long contexts without token limit constraints. Experimental results demonstrate the effectiveness of our approach in all types of long document classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。