低资源语言依赖句法分析中，传统LSTM反超Transformer模型。
Dependency Parsing Across the Resource Spectrum: Evaluating Architectures on High and Low-Resource Languages
- 用Biaffine LSTM、Stack-Pointer等四类模型跨12种语言对比分析
- 在低资源语料下，Biaffine LSTM性能优于Transformer，数据量大时后者才胜出
- 形态复杂度是影响Transformer表现的关键因素，适合低资源场景研究者参考
基于Transformer的模型在高资源语言上达到依赖句法分析的最先进水平，但在低资源环境下对简单架构的优势仍不明确。我们评估了四种解析器——Biaffine LSTM、Stack-Pointer Network、AfroXLMR-large和RemBERT——在12种类型多样语言上的表现，重点考察低资源非洲语言。结果发现，在低资源条件下，Biaffine LSTM始终优于Transformer模型；随着训练数据增加，Transformer才逐步恢复优势。这一转折点出现在典型低资源语料库的数据规模范围内。在控制语料规模后，形态复杂度（通过MATTR衡量）成为影响Transformer相对劣势的显著次级预测因子。结果表明，在缺乏足够标注数据前，Biaffine LSTM可能更适合低资源语境下的语法工具开发。
原文摘要 · Abstract (English)
Transformer-based models achieve state-of-the-art dependency parsing for high-resource languages, yet their advantage over simpler architectures in low-resource settings remains poorly understood. We evaluate four parsers---the Biaffine LSTM, Stack-Pointer Network, AfroXLMR-large, and RemBERT---across twelve typologically diverse languages, with a focus on low-resource African languages. We find that the Biaffine LSTM consistently outperforms transformer models in low-resource regimes, with transformers recovering their advantage as training data increases. The crossover falls within a resource range typical of treebanks for under-resourced languages. Morphological complexity (measured via MATTR) emerges as a significant secondary predictor of transformers' relative disadvantage after controlling for corpus size. These results indicate that the Biaffine LSTM may be better suited for syntactic tool development in low-resource regimes until sufficient annotated data is available to leverage the representational capacity of pre-trained transformers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。