提出局部语法消歧方法,确保词性标注不遗漏正确标签。
Levée d'ambiguïtés par grammaires locales
- 基于局部语法规则,结合上下文消除词性歧义
- 验证语法时需考虑规则间的交互作用,不可孤立分析
- 适合对标注完整性要求高的自然语言处理任务
词汇的词性(POS)常具有歧义,但在具体文本中歧义通常可被上下文缓解。词性消歧即利用上下文减少词汇可能的词性数量,是词法标注的核心挑战之一。该问题广泛存在于拼写纠错、语法检查、表达识别、语音合成及语料库分析等自然语言处理任务中。近年来的词法标注系统在遇到词汇歧义或无法确定唯一正确标签时,常输出多个候选解。本研究聚焦于实现零漏检率的目标——确保正确词性绝不被排除。此目标对单一标签系统不现实。本文介绍一种适配零漏检率目标的词性消歧方法,源自Silberztein的INTEX系统(1993)。我们给出该方法的形式化描述,并指出:在该框架下,仅分别验证转换器路径是不够的,必须考察路径间的相互作用;若使用多个转换器组合,其整体效果也无法通过独立分析预测。此外,初始标注中显现出的消歧规则虽看似合理,但可能因未预见的句法结构或歧义而失效。因此,在追求零漏检率时,必须对局部语法进行严格测试,详细预设其在实际文本中的行为至关重要。
原文摘要 · Abstract (English)
Many words are ambiguous in terms of their part of speech (POS). However, when a word appears in a text, this ambiguity is generally much reduced. Disambiguating POS involves using context to reduce the number of POS associated with words, and is one of the main challenges of lexical tagging. The problem of labeling words by POS frequently arises in natural language processing, for example for spelling correction, grammar or style checking, expression recognition, text-to-speech conversion, text corpus analysis, etc. Lexical tagging systems are thus useful as an initial component of many natural language processing systems. A number of recent lexical tagging systems produce multiple solutions when the text is lexically ambiguous or the uniquely correct solution cannot be found. These contributions aim to guarantee a zero silence rate: the correct tag(s) for a word must never be discarded. This objective is unrealistic for systems that tag each word uniquely. This article concerns a lexical disambiguation method adapted to the objective of a zero silence rate and implemented in Silberztein's INTEX system (1993). We present here a formal description of this method. We show that to verify a local disambiguation grammar in this framework, it is not sufficient to consider the transducer paths separately: one needs to verify their interactions. Similarly, if a combination of multiple transducers is used, the result cannot be predicted by considering them in isolation. Furthermore, when examining the initial labeling of a text as produced by INTEX, ideas for disambiguation rules come spontaneously, but grammatical intuitions may turn out to be inaccurate, often due to an unforeseen construction or ambiguity. If a zero silence rate is targeted, local grammars must be carefully tested. This is where a detailed specification of what a grammar will do once applied to texts would be necessary.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。