arXiv:2511.23088cs.CL2025-11

为古老梵文《梨俱吠陀》开发自动加音系统,提升古籍数字化可读性。

Accent Placement Models for Rigvedic Sanskrit Text

  • 用Byte-level Transformer和LoRA微调实现音调标记自动修复
  • 全量微调ByT5误差最低,LoRA在效率与精度间取得平衡
  • 提出分离字形与音调错误的评估方法,适合古语文本研究者

《梨俱吠陀》作为最古老的印度梵语文本之一,采用独特的声调系统(udātta、anudātta、svarita),其音调符号常缺失于现代电子文本。本文构建了带音调与无音调诗句的平行语料库,对比三种自动加音策略:(i) 全量微调ByT5(基于字节的Transformer,直接处理Unicode组合标记),(ii) 从零训练的BiLSTM-CRF序列标注基线,(iii) 基于LoRA的参数高效微调。评估使用词错误率(WER)、字符错误率(CER)及专用于音调修正的标点错误率(DER)。结果显示,全量微调ByT5在所有指标上表现最佳;LoRA在精度与效率间表现优异;BiLSTM-CRF提供透明基线。研究强调音调恢复需具备Unicode安全预处理、标记感知分词及区分字形与音调错误的评估机制,推动遗产语言技术发展,为音调感知的OCR、语音合成与数字人文研究建立可复现基准。

原文摘要 · Abstract (English)

The Rigveda, among the oldest Indian texts in Vedic Sanskrit, employs a distinctive pitch-accent system : udātta, anudātta, svarita whose marks encode melodic and interpretive cues but are often absent from modern e-texts. This work develops a parallel corpus of accented-unaccented ślokas and conducts a controlled comparison of three strategies for automatic accent placement in Rigvedic verse: (i) full fine-tuning of ByT5, a byte-level Transformer that operates directly on Unicode combining marks, (ii) a from-scratch BiLSTM-CRF sequence-labeling baseline, and (iii) LoRA-based parameter-efficient fine-tuning atop ByT5. Evaluation uses Word Error Rate (WER) and Character Error Rate (CER) for orthographic fidelity, plus a task-specific Diacritic Error Rate (DER) that isolates accent edits. Full ByT5 fine-tuning attains the lowest error across all metrics; LoRA offers strong efficiency-accuracy trade-offs, and BiLSTM-CRF serves as a transparent baseline. The study underscores practical requirements for accent restoration - Unicode-safe preprocessing, mark-aware tokenization, and evaluation that separates grapheme from accent errors - and positions heritage-language technology as an emerging NLP area connecting computational modeling with philological and pedagogical aims. Results establish reproducible baselines for Rigvedic accent restoration and provide guidance for downstream tasks such as accent-aware OCR, ASR/chant synthesis, and digital scholarship.

古语文本音调标注TransformerLoRA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。