arXiv:2510.03781cs.CLcs.AI2025-10被引 6

用AI构建120万条圣训语料库,实现多语言、结构化处理。

Rezwan: Leveraging Large Language Models for Comprehensive Hadith Text Processing: A 1.2M Corpus Development

  • 用大模型自动分割、清洗和增强圣训文本,形成可研究语料
  • 人工评估显示关键任务准确率达9.33/10,整体质量优于人工库
  • 成本仅为传统方法的极小部分,适合大规模宗教文本数字化

本文提出Rezwan,一个包含超过120万条圣训的大型人工智能辅助语料库,通过全自动流程从Maktabat Ahl al-Bayt等数字资源中提取并结构化。该流程利用大语言模型(LLMs)完成分段、链式文本分离、验证及多层增强,每条圣训均配备12种语言机器翻译、智能变音标注、摘要生成、主题标记和跨文本语义分析。经过对1,213条随机抽样圣训的六位领域专家评估,链式文本分离与摘要任务准确率分别达9.33/10,但变音标注与语义相似度检测仍存挑战。相较人工标注的Noor Corpus,Rezwan在规模与质量上均更优,平均得分8.46/10对3.66/10。成本分析表明,原需超22.9万小时专家工作量的任务,在数月内以极低成本完成。该研究展示了AI如何增强人类专业能力,推动伊斯兰文化遗产的多语言、智能化研究。

原文摘要 · Abstract (English)

This paper presents the development of Rezwan, a large-scale AI-assisted Hadith corpus comprising over 1.2M narrations, extracted and structured through a fully automated pipeline. Building on digital repositories such as Maktabat Ahl al-Bayt, the pipeline employs Large Language Models (LLMs) for segmentation, chain--text separation, validation, and multi-layer enrichment. Each narration is enhanced with machine translation into twelve languages, intelligent diacritization, abstractive summarization, thematic tagging, and cross-text semantic analysis. This multi-step process transforms raw text into a richly annotated research-ready infrastructure for digital humanities and Islamic studies. A rigorous evaluation was conducted on 1,213 randomly sampled narrations, assessed by six domain experts. Results show near-human accuracy in structured tasks such as chain--text separation (9.33/10) and summarization (9.33/10), while highlighting ongoing challenges in diacritization and semantic similarity detection. Comparative analysis against the manually curated Noor Corpus demonstrates the superiority of Najm in both scale and quality, with a mean overall score of 8.46/10 versus 3.66/10. Furthermore, cost analysis confirms the economic feasibility of the AI approach: tasks requiring over 229,000 hours of expert labor were completed within months at a fraction of the cost. The work introduces a new paradigm in religious text processing by showing how AI can augment human expertise, enabling large-scale, multilingual, and semantically enriched access to Islamic heritage.

圣训处理大模型应用多语言数字人文

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。