arXiv:2602.15139cs.CLcs.AI2026-02

针对圣训文本问答难题,提出概念引导的高效增强模型。

CGRA-DeBERTa Concept Guided Residual Augmentation Transformer for Theologically Islamic Understanding

  • 用概念字典引导残差块,动态强化关键语义词。
  • 在圣训数据集上达97.85%精确率,超越DeBERTa 8.08个百分点。
  • 适合需要精准神学理解的宗教文本智能系统开发者。

经典伊斯兰文本的精准问答仍面临领域语义复杂、长程依赖和概念敏感推理的挑战。为此,本文提出CGRA-DeBERTa——一种概念引导的残差增强变换器框架,用于提升对圣训文献的神学问答性能。该模型基于定制化DeBERTa主干网络,采用轻量级LoRA适配与残差概念感知门控机制。定制嵌入模块捕捉全局与位置上下文,概念引导残差块融合12个核心神学概念的先验知识。概念门控机制通过重要性加权注意力,对关键标记进行1.04至3.00倍差异性缩放,既保持上下文完整性,又强化领域语义表示,并实现高精度、高效的跨度抽取,同时维持计算效率。实验使用从《布哈里圣训集》与《穆斯林圣训集》构建的42,591对问答数据集训练。对比Bert(EM=75.87)与DeBERTa(EM=89.77),本模型达到97.85的精确率,绝对提升8.08,推理开销仅增加约8%。定性评估显示其提取更准确、区分更清晰、神学表达更精确。本研究为教育材料提供高效、可解释且精准的圣训问答系统。

原文摘要 · Abstract (English)

Accurate QA over classical Islamic texts remains challenging due to domain specific semantics, long context dependencies, and concept sensitive reasoning. Therefore, a new CGRA DeBERTa, a concept guided residual domain augmentation transformer framework, is proposed that enhances theological QA over Hadith corpora. The CGRA DeBERTa builds on a customized DeBERTa transformer backbone with lightweight LoRA based adaptations and a residual concept aware gating mechanism. The customized DeBERTa embedding block learns global and positional context, while Concept Guided Residual Blocks incorporate theological priors from a curated Islamic Concept Dictionary of 12 core terms. Moreover, the Concept Gating Mechanism selectively amplifies semantically critical tokens via importance weighted attention, applying differential scaling from 1.04 to 3.00. This design preserves contextual integrity, strengthens domain-specific semantic representations, and enables accurate, efficient span extraction while maintaining computational efficiency. This paper reports the results of training CGRA using a specially constructed dataset of 42591 QA pairs from the text of Sahih alBukhari and Sahih Muslim. While BERT achieved an EM score of 75.87 and DeBERTa one of 89.77, our model scored 97.85 and thus surpassed them by 8.08 on an absolute scale, all while adding approximately 8 inference overhead due to parameter efficient gating. The qualitative evaluation noted better extraction and discrimination and theological precision. This study presents Hadith QA systems that are efficient, interpretable, and accurate and that scale provide educational materials with necessary theological nuance.

神学问答圣训理解概念引导轻量化模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。