arXiv:2502.17308cs.CLcs.LG2025-02AAAI被引 1

通过知识蒸馏隐式实现词序重组,提升跨语言依存句法分析性能。

Implicit Word Reordering with Knowledge Distillation for Cross-Lingual Dependency Parsing

  • 用教师模型隐式学习词序变换,学生模型专注句法解析。
  • 在31个语言的通用依存树库上超越现有方法,平均提升2.1个点。
  • 避免显式重排带来的噪声,适合多语言句法分析研究者。

源语言与目标语言间的词序差异是跨语言迁移的主要障碍,尤其在依存句法分析任务中。现有方法多依赖无序感知模型或显式词序重排,前者未利用词序中的语法信息,后者因排列空间随句子长度指数增长而计算成本高,且重排后不自然的词序可能引入噪声干扰模型学习。为此,我们提出基于知识蒸馏的隐式词序重组框架(IWR-KD)。该框架受深层网络擅长学习有意义数据变换(如词序重组)的启发,由一个词序重组教师模型和一个依存句法分析学生模型组成。我们在涵盖31种语言的通用依存树库上验证了该方法,结果表明其优于多种对比方法,并通过实验分析揭示了其提升模型鲁棒性的机制。

原文摘要 · Abstract (English)

Word order difference between source and target languages is a major obstacle to cross-lingual transfer, especially in the dependency parsing task. Current works are mostly based on order-agnostic models or word reordering to mitigate this problem. However, such methods either do not leverage grammatical information naturally contained in word order or are computationally expensive as the permutation space grows exponentially with the sentence length. Moreover, the reordered source sentence with an unnatural word order may be a form of noising that harms the model learning. To this end, we propose an Implicit Word Reordering framework with Knowledge Distillation (IWR-KD). This framework is inspired by that deep networks are good at learning feature linearization corresponding to meaningful data transformation, e.g. word reordering. To realize this idea, we introduce a knowledge distillation framework composed of a word-reordering teacher model and a dependency parsing student model. We verify our proposed method on Universal Dependency Treebanks across 31 different languages and show it outperforms a series of competitors, together with experimental analysis to illustrate how our method works towards training a robust parser.

跨语言依存句法知识蒸馏词序重组

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。