测试中文NLP模型对形构歧义的鲁棒性，发现分词错误普遍且影响情感分析。
ERAS: Evaluating the Robustness of Chinese NLP Models to Morphological Garden Path Errors
- 构建ERAS基准，对比含歧义与无歧义分词句子的表现差异。
- 分词模型在局部歧义句中出错，但非歧义句表现正常。
- 字符级模型虽无显式分词，仍隐含形构歧义错误，适合评估模型上下文理解力。
在无词边界标记的语言中，NLP模型需执行词分段，或作为预处理，或作为端到端计算的隐式步骤。本文揭示中文NLP模型易受形构歧义错误影响：因未能利用句级形态句法上下文解决局部分词歧义。为此提出基准ERAS，通过对比含与不含局部分词歧义句子中模型行为，评估其脆弱性。实验表明，分词模型在局部歧义句中出现形构歧义错误，但在无歧义句中不犯类似错误。进一步发现，采用字符级分词的情感分析模型即使无显式分词步骤，也会产生隐式形构歧义错误。结果表明，当前中文文本分词常无法充分考虑形态句法上下文。
原文摘要 · Abstract (English)
In languages without orthographic word boundaries, NLP models perform word segmentation, either as an explicit preprocessing step or as an implicit step in an end-to-end computation. This paper shows that Chinese NLP models are vulnerable to morphological garden path errors: errors caused by a failure to resolve local word segmentation ambiguities using sentence-level morphosyntactic context. We propose a benchmark, ERAS, that tests a model's vulnerability to morphological garden path errors by comparing its behavior on sentences with and without local segmentation ambiguities. Using ERAS, we show that word segmentation models make garden path errors on locally ambiguous sentences, but do not make equivalent errors on unambiguous sentences. We further show that sentiment analysis models with character-level tokenization make implicit garden path errors, even without an explicit word segmentation step in the pipeline. Our results indicate that models' segmentation of Chinese text often fails to account for morphosyntactic context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。