arXiv:2602.20749cs.CL2026-02被引 2

用显式语法特征融合提升文本分类鲁棒性,轻量高效适合边缘设备。

Explicit Grammar Semantic Feature Fusion for Robust Text Classification

  • 将句法结构显式编码为紧凑语法向量,与冻结上下文嵌入融合
  • 在多个数据集上比基线模型提升2%-15%,跨领域泛化能力强
  • 无需全参数微调或复杂架构,适合资源受限环境部署

自然语言处理通过分析和分类文本实现计算机对人类语言的理解,现有模型依赖大规模语料训练的Transformer架构,计算开销大,不适用于资源受限环境。本文提出一种结合全面语法规则与语义信息的轻量级分类模型,避免使用完整参数化的Transformer或复杂深度学习结构。其创新在于显式编码句子级语法结构(包括句法组成、短语模式和复杂度指标)为紧凑语法向量,并与冻结的上下文嵌入融合,形成统一表示,同时捕捉文本的结构与语义特征。采用深度信念网络(DBNs)、长短期记忆网络(LSTMs)、双向LSTM(BiLSTMs)以及基于Transformer的BERT和XLNet进行训练与评估,实验表明该统一特征表示模型在多个数据集上相较基线模型性能提升2%-15%,支持跨异构领域的有效学习。与以往通过额外注意力层、树编码器或全微调注入语法结构的方法不同,本框架将语法作为显式归纳偏置,而非可学习模块,从而实现极轻量模型,在边缘设备上表现更优。

原文摘要 · Abstract (English)

Natural Language Processing enables computers to understand human language by analysing and classifying text efficiently with deep-level grammatical and semantic features. Existing models capture features by learning from large corpora with transformer models, which are computationally intensive and unsuitable for resource-constrained environments. Therefore, our proposed study incorporates comprehensive grammatical rules alongside semantic information to build a robust, lightweight classification model without resorting to full parameterised transformer models or heavy deep learning architectures. The novelty of our approach lies in its explicit encoding of sentence-level grammatical structure, including syntactic composition, phrase patterns, and complexity indicators, into a compact grammar vector, which is then fused with frozen contextual embeddings. These heterogeneous elements unified a single representation that captures both the structural and semantic characteristics of the text. Deep learning models such as Deep Belief Networks (DBNs), Long Short-Term Memory (LSTMs), BiLSTMs, and transformer-based BERT and XLNET were used to train and evaluate the model, with the number of epochs varied. Based on experimental results, the unified feature representation model captures both the semantic and structural properties of text, outperforming baseline models by 2%-15%, enabling more effective learning across heterogeneous domains. Unlike prior syntax-aware transformer models that inject grammatical structure through additional attention layers, tree encoders, or full fine-tuning, the proposed framework treats grammar as an explicit inductive bias rather than a learnable module, resulting in a very lightweight model that delivers better performance on edge devices

文本分类语法融合轻量化边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。