arXiv:2510.11537cs.CL2025-10

用图注意力增强PhoBERT,提升越南语词级分类效果

An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification

  • 将PhoBERT嵌入与图注意力结合,捕捉词间深层依赖
  • 在3个越语数据集上均超越基线模型,最高提升2.1个点
  • 特别适合医疗、语音等专业领域中文本标注任务

我们提出一种新型神经架构TextGraphFuseGAT,将预训练Transformer编码器(PhoBERT)与图注意力网络(GAT)融合,用于词级分类任务。该模型在PhoBERT生成的词嵌入基础上构建全连接图,使GAT层能捕获序列上下文之外的丰富词间依赖关系。为进一步增强上下文表示,在图增强嵌入上应用Transformer式自注意力层。最终的词表示通过分类头完成序列标注。我们在三个越语基准数据集上评估:PhoNER-COVID19(新冠领域命名实体识别)、PhoDisfluency(语音不流畅检测)和VietMed-NER(首个越语医疗口语命名实体识别数据集),后者包含18类实体,源自真实医疗语音转录文本并采用BIO标注。其专业词汇与领域表达使其成为极具挑战性的基准。实验结果表明,该方法在多个领域持续优于强基线模型,包括纯Transformer和混合神经模型(如BiLSTM + CNN + CRF),验证了结合预训练语义特征与图关系建模对词级分类的有效性。

原文摘要 · Abstract (English)

We propose a novel neural architecture named TextGraphFuseGAT, which integrates a pretrained transformer encoder (PhoBERT) with Graph Attention Networks for token-level classification tasks. The proposed model constructs a fully connected graph over the token embeddings produced by PhoBERT, enabling the GAT layer to capture rich inter-token dependencies beyond those modeled by sequential context alone. To further enhance contextualization, a Transformer-style self-attention layer is applied on top of the graph-enhanced embeddings. The final token representations are passed through a classification head to perform sequence labeling. We evaluate our approach on three Vietnamese benchmark datasets: PhoNER-COVID19 for named entity recognition in the COVID-19 domain, PhoDisfluency for speech disfluency detection, and VietMed-NER for medical-domain NER. VietMed-NER is the first Vietnamese medical spoken NER dataset, featuring 18 entity types collected from real-world medical speech transcripts and annotated with the BIO tagging scheme. Its specialized vocabulary and domain-specific expressions make it a challenging benchmark for token-level classification models. Experimental results show that our method consistently outperforms strong baselines, including transformer-only and hybrid neural models such as BiLSTM + CNN + CRF, confirming the effectiveness of combining pretrained semantic features with graph-based relational modeling for improved token classification across multiple domains.

越南语图神经网络命名实体识别PhoBERT

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。