用动态窗口注意力构建文档图结构，提升长文本分类与摘要效果
From Global to Local: Learning Context-Aware Graph Representations for Document Classification and Summarization
- 基于滑动窗口注意力自动构建文档图，捕捉句间语义关系
- 在分类任务上表现接近最优，计算开销更低
- 适合长文档处理，对摘要生成有探索性潜力
当前NLP系统通常将文档表示为线性词元序列，虽能保留顺序信息，但难以建模长距离依赖和全局文档结构，尤其在长文本中。本文提出一种数据驱动的方法，自动构建基于图的文档表示。基于Bugueño与de Melo（2025）的工作，我们采用动态滑动窗口注意力模块，有效捕捉句子间的局部与中程语义依赖以及文档内部的结构关系。在所构建图上训练的图注意力网络（GATs）在文档分类任务上取得具有竞争力的结果，且所需计算资源低于以往方法。我们还对所提图构建方法在抽取式文档摘要中的应用进行了探索性评估，揭示其潜力与现有局限。项目实现已开源。
原文摘要 · Abstract (English)
Recent NLP systems commonly represent documents as linear token sequences. Although this captures sequential order, it can hinder modeling long-range dependencies and global document structure, especially for long texts. This paper proposes a data-driven method to automatically construct graph-based document representations. Building upon the recent work of Bugueño and de Melo (2025), we leverage the dynamic sliding-window attention module to effectively capture local and mid-range semantic dependencies between sentences, as well as structural relations within documents. Graph Attention Networks (GATs) trained on our learned graphs achieve competitive results on document classification while requiring lower computational resources than previous approaches. We further present an exploratory evaluation of the proposed graph construction method for extractive document summarization, highlighting both its potential and current limitations. The implementation of this project can be found on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。