首次量化分析中亚东干语形态特征,揭示其语法核心极为简洁。
The Morphological Core of Dungan: A Two-Dialect Finite-State Model and a Multi-Genre Evaluation
- 用有限状态分析器形式化东干语形态知识,覆盖甘、陕两方言。
- 仅9.3%词有显性词尾,95%未识别词因词汇缺失而非形态复杂。
- 开源工具与评估脚本,适合语言学与自然语言处理研究者。
东干语是中亚使用的汉语方言,使用西里尔字母书写。尽管其语法在文献中已有详述,但其实际使用中的形态定量特征尚未系统测量。本文使用基于HFST的有限状态形态分析器,涵盖甘南和陕西两种方言变体,不提出新语法描述,而是形式化已有研究成果,并在三种文本类型语料上实现可测量性。结果表明:显性屈折罕见且受限——百科类文本中仅9.3%的词具有显性标记,系统仅有10个范畴,程度标记几乎不存在;歧义真实但高度集中——78.1%的词仅有一个分析,剩余部分几乎全部集中在两个虚词- di(属格/进行体)和- ni(处所/将来体);语法核心基本封闭——78%至95%的未识别词因不在词库中而无法处理,模型刻意忽略的现象仅占失败样本的4.5%。保留测试覆盖率(80%-85%)不低于训练覆盖率(73%),无形态词表已达67.4%,故形态贡献5.2分。东干语的开放前沿在于词汇。分析器、源码及所有评估脚本均公开发布。
原文摘要 · Abstract (English)
Dungan, a Sinitic language of Central Asia written in a Cyrillic-based script, is described in detail in the grammatical literature, yet the quantitative properties of its morphology in actual usage have, to the best of our knowledge, never been measured systematically. This paper uses a finite-state morphological analyzer as a measuring instrument. Implemented with HFST and covering both dialect groups (the Gansu variety, which is the literary standard, and the Shaanxi variety), the model offers no new grammatical description; it formalises the knowledge accumulated in Dungan studies and makes it measurable on corpora of three genres. Three results follow. Overt inflection is rare and limited: only 9.3% of recognized tokens in the encyclopaedic register have an overt marker, the system has just ten categories, and degree marking is almost absent. Ambiguity is genuine but sharply localized: 78.1% of tokens receive a single analysis, and the residue sits almost entirely on two clitics, -di (genitive/progressive) and -ni (locative/prospective). And the grammatical core proves effectively closed, the claim the instrument is really needed for: between 78% and 95% of the tokens the analyzer fails on, depending on register, are simply absent from the lexicon, and the phenomena the model deliberately declines to implement account for at most 4.5% of those failures. Held-out coverage (80 to 85%) is no lower than development coverage (73%), while a stem list with no morphology already reaches 67.4%, so the morphology is worth 5.2 points. The open frontier of Dungan is lexical. The analyzer, its sources and every evaluation script are released openly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。