用预训练编码器-解码器模型做句法解析，效果优于现有序列模型。
Exploiting Pre-trained Encoder-Decoder Transformers for Sequence-to-Sequence Constituent Parsing
- 基于BART、mBART、T5等预训练编码器-解码器构建句法解析器
- 在连续和非连续树库上均超越已有序列模型，接近专用解析器性能
- 验证了线性化策略对复杂句法结构的有效性，适合自然语言理解任务
为实现深度自然语言理解，句法成分解析至关重要，广泛应用于文本与语音处理的人工智能系统。近期方法将成分解析视为机器翻译问题，采用标准序列到序列模型，摆脱传统任务特定解析器。这些模型通常以BERT或RoBERTa等预训练编码器模型初始化。然而，预训练编码器-解码器模型在成分解析中的应用尚未充分探索。为此，我们扩展序列到序列框架，研究基于BART、mBART和T5等预训练编码器-解码器架构的解析器，通过微调生成线性化句法树，并在连续与复杂非连续基准上系统评估不同线性化策略。结果表明，该方法优于所有先前序列模型，在连续成分解析上表现可与领先任务特定解析器媲美。
原文摘要 · Abstract (English)
To achieve deep natural language understanding, syntactic constituent parsing plays a crucial role and is widely required by many artificial intelligence systems for processing both text and speech. A recent approach involves using standard sequence-to-sequence models to handle constituent parsing as a machine translation problem, moving away from traditional task-specific parsers. These models are typically initialized with pre-trained encoder-only language models like BERT or RoBERTa. However, the use of pre-trained encoder-decoder language models for constituency parsing has not been thoroughly explored. To bridge this gap, we extend the sequence-to-sequence framework by investigating parsers built on pre-trained encoder-decoder architectures, including BART, mBART, and T5. We fine-tune them to generate linearized parse trees and extensively evaluate them on different linearization strategies across both continuous treebanks and more complex discontinuous benchmarks. Our results demonstrate that our approach outperforms all prior sequence-to-sequence models and performs competitively with leading task-specific constituent parsers on continuous constituent parsing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。