arXiv:2411.04822cs.CL2024-11被引 3

跨语言迁移未必有效,古汉语文本难助韩日古籍处理

Shared Heritage, Distinct Writing: Rethinking Resource Selection for East Asian Historical Documents

  • 用古汉语资源迁移至韩日古文,实际效果有限
  • 韩文古籍在迁移后性能提升不足±0.0068 F1,翻译仅+0.84 BLEU
  • 仅在极低资源场景下有明显帮助,需实证验证迁移可行性

汉文化圈历史文献共享相似格式与书写传统,尤其体现在宫廷史官编纂的实录中。研究者常将古汉语资源用于跨语言迁移,以支持韩、日等低资源历史文献处理。本文质疑从古汉语向韩文汉字(Hanja)和日文汉文(Kanbun)迁移的有效性。在机器翻译、命名实体识别与标点恢复任务中,实验显示古汉语数据对韩文古籍模型性能影响极小:序列标注任务F1-score差异在±0.0068内,翻译任务最高仅提升0.84 BLEU。该现象在不同模型规模、架构及领域数据集下均持续存在。分析表明,随着本地语料增加,古汉语资源的收益迅速衰减;仅在极端低资源条件下,韩、日古籍处理才显著受益。研究强调应基于实证而非假设进行跨语言迁移选择。

原文摘要 · Abstract (English)

Historical documents in the Sinosphere are known to share common formats and practices, particularly in veritable records compiled by court historians. This shared linguistic heritage has led researchers to use Classical Chinese resources for cross-lingual transfer when processing historical documents from Korea and Japan, which remain relatively low-resource. In this paper, we question the assumption of cross-lingual transferability from Classical Chinese to Hanja and Kanbun, the ancient written languages of Korea and Japan, respectively. Our experiments across machine translation, named entity recognition, and punctuation restoration tasks show minimal impact of Classical Chinese datasets on language model performance for ancient Korean documents written in Hanja, with performance differences within $\pm{}0.0068$ F1-score for sequence labeling tasks and up to $+0.84$ BLEU score for translation. These limitations persist consistently across various model sizes, architectures, and domain-specific datasets. Our analysis reveals that the benefits of Classical Chinese resources diminish rapidly as local language data increases for Hanja, while showing substantial improvements only in extremely low-resource scenarios for both Korean and Japanese historical documents. These findings emphasize the need for careful empirical validation rather than assuming benefits from indiscriminate cross-lingual transfer.

古籍处理跨语言迁移低资源东亚文献

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。